Add pause/resume and auto resume to the offline API

[itaykinnrot]

Today if you try to download a big movie and stop in the middle - next time you'll need to restart the download since there isn't any logic that check what already downloaded.
The suggestion here is to check if the keys already stored and only fetch the missing ones.
In addition user may want to pause & resume download - add this API

[TheModMaker]

We'll consider it in the future. This will likely just have to be an explicit pause() and resume() methods, we may not be able to recover from a stopped download (e.g. by closing the tab). This is because it will be difficult or impossible to tell the difference between a download that stopped and one that is still going in another tab.

But we may be able to have an explicit pause method where we can store which pieces we have downloaded already. But this will be a low priority feature since we would need to store much more info in the database and we are already having troubles with different database versions.

Note that we can't just detect existing segments since: (a) we use unique keys so there will never be existing keys and (b) a segment that is the same as an existing one may be from another stream so we can't just reuse it. Currently we delete segments when we delete the manifest, so if we reused the segment, that segment would disappear when we deleted the other manifest.

Also note that if you stop a download using storage.destroy(), then we will delete any partially downloaded content, so there is no resuming.

[OrenMe]

Pause/resume of a download without closing the tab is indeed easy - already have a working POC with a checkPaused promise before calling download segment.
I think this is very narrow use case as clients would like a more rich experience.
Can you elaborate on the DB issues? Is this the only limitation as you currently see it? As I think it would be indeed mandatory to signal the download state between tabs via the DB with a manifest download state flag or some kind of signal.

[TheModMaker]

See #1248 for more info on DB issues. Basically we are having troubles with database versions and upgrades. We will need to make some major changes to the internals to allow for handling different database versions. Since this will require more changes to database structure including possible changes to the schema itself (e.g. maybe adding a new table for pending segments), I think this should wait for that issue to be closed.

[joeyparrish]

We will also need to consider how pause/resume interacts with background fetch in #879.

[odedhutzler]

Hey @joeyparrish, any updates here? Are you planning to add the new features?

[OrenMe]

Hi @joeyparrish @TheModMaker is there any update to pause/resume capabilities?

[OrenMe]

Hi, any update on this?

[joeyparrish]

Hi, any update on this?

We are not working on this internally at the moment.

Are you planning to add the new features?

Yes, Shaka Player development is still ongoing. We are still adding features and making releases.

[OrenMe]

thanks @joeyparrish, will you be open for us to work on it and contribute back? wouldn't want to start work on this without syncing with you first.

[joeyparrish]

@OrenMe, sure, that sounds good to me. We'd love to have a PR for this. Just give us an example of the API you're adding, so we can make sure it's in line with our own designs. Better to give you feedback on the API before you get all the way to a PR, so you don't waste time.

[loicraux]

Hi, any update on this?

This is a feature I'd be interested in, also...

[joeyparrish]

No, no updates. You can see the last comment was from 2019, when I said we would love to have a PR for this. It's still true! We (the Shaka team at Google) are not working on this, and it is not a priority for us currently. However, this is open source, and the community can and should contribute to push forward the things that matter to them!

Let us know if you'd like to work on it, and I can assign the issue to you. We can also provide guidance if you get stuck. Thanks!

[loicraux]

No, no updates. You can see the last comment was from 2019, when I said we would love to have a PR for this. It's still true! We (the Shaka team at Google) are not working on this, and it is not a priority for us currently. However, this is open source, and the community can and should contribute to push forward the things that matter to them!

Let us know if you'd like to work on it, and I can assign the issue to you. We can also provide guidance if you get stuck. Thanks!

If this becomes a necessity on my part, and if I'm given the necessary time, I'd of course be happy to propose a PR to add this functionality.

The difficulty for me would mainly concern the design of the shaka player code, in order to first propose an acceptable solution before spending time on its implementation. I'm comfortable with unit tests, builds, the PR process, etc... but it's understanding the design of shaka player that could take me longer and pose more difficulties. I see that there are design documents under docs/design folder but these documents are between 2 and 7 years old, so I'm not sure if they're up to date when I read them...

To take an example, I can only see the graph data struct + visitor pattern is extensively used. I am familiar with the pattern, but not with the contents of the nodes and transitions/edges in shaka player... Do you think it would help to somehow triage the design docs under docs/design as docs/design/outdated (or docs/design/legacy ?), docs/design/uptodate and docs/design/ongoing_work, so that getting familiar with the current design gets easier ?

[joeyparrish]

Yes, you are right. The design docs folder needs to be cleaned up. Feel free to file a new issue for that!

[rezaan6]

@joeyparrish , came across the issue thread. Do we have a solution for offline api resume download in the current version? Or do you know an alternative solution, be that as a custom one or an entire new npm lib player.

[joeyparrish]

@rezaan6, this issue is still open, so nobody has added that API yet. Please contribute if you're interested!

Or do you know an alternative solution, be that as a custom one or an entire new npm lib player.

I am biased, but I would prefer you not create an entirely new player to solve this issue...