Transmux in a worker

[Kogoruhn]

Have you read the FAQ and checked for duplicate open issues?:
Yes
What version of Shaka Player are you using?:
2.5.0-beta2
Can you reproduce the issue with our latest release version?:
Yes
Can you reproduce the issue with the latest code from master?:
Yes
Are you using the demo app or your own custom app?:
custom
If custom app, can you reproduce the issue using our demo app?:
not applicable
What browser and OS are you using?:
webOS/Tizen
What are the manifest and license server URIs?:
What did you do?
try to play various hls streams
What did you expect to happen?
normal playback
What actually happened?
WebOS: video plays 0.5-2-10 sec, then freezing on 0.5-2-20 sec (depends on stream);
Tizen: barely noticeable freezings of playback.

Streams are not buggy - BBC TAL HTML5 player & videojs plays these hls on these smarttv-s normally.
This issue a bit like that, gap jumping fires time after time (and Flushing media pipeline too)

player.getConfiguration().streaming:

alwaysStreamText: false,
bufferBehind: 3,
bufferingGoal: 10,
durationBackoff: 1,
forceTransmuxTS: true,
ignoreTextStreamFailures: false,
jumpLargeGaps: false,
rebufferingGoal: 2,
safeSeekOffset: 5,
smallGapLimit: 0.5,
startAtSegmentBoundary: false

logs (use horisontal scroll at the bottom)

Guessing stream type for #EXT-X-STREAM-INF:CLOSED-CAPTIONS="NONE",RESOLUTION="1920x1080",FRAME-RATE="25.000",CODECS="avc1.4d0028,mp4a.40.2",AVERAGE-BANDWIDTH="1741137",BANDWIDTH="2176421",URI="tracks-v1a1/mono.m3u8" hls_parser.js:604
Guessing multiplexed audio+video. hls_parser.js:630
XMLHttpRequest cannot load https://brics.bonus-tv.ru/cdn/brics/tracks-v1a1/2018/12/18/12/22/17-04960.ts. Request header field range is not allowed by Access-Control-Allow-Headers. ?duid=e57cd6cd-6581-fd59-6411-616d33f5219e:1
Updating manifest... hls_parser.js:2142
XMLHttpRequest cannot load https://brics.bonus-tv.ru/cdn/brics/tracks-v1a1/2018/12/18/12/22/17-04960.ts. Request header field range is not allowed by Access-Control-Allow-Headers. ?duid=e57cd6cd-6581-fd59-6411-616d33f5219e:1
Unable to fetch a partial HLS segment! Falling back to a full segment request, which is expensive!  Your server should support Range requests and CORS preflights. https://brics.bonus-tv.ru/cdn/brics/tracks-v1a1/2018/12/18/12/22/17-04960.ts hls_parser.js:1468
First segment 17-04960.ts starts at 92835.83826666667 hls_parser.js:1403
Found variant with audio and video content, so filtering out audio-only content in all periods. player.js:717
codecs avc1- avg bandwidth 2176421 player.js:1021
onChooseStreams_ Object {startTime: 0, variants: Array[1], textStreams: Array[0]} player.js:2939
Choosing new streams after period changed player.js:2987
init: completed initial Stream setup streaming_engine.js:439
No preferred audio language set.  We will choose an arbitrary language initially player.js:923
The video has now been LOADED drm-player.js?v=1.2:81
(video:1) looking up segment: presentationTime=92838.68726682663 currentPeriod.startTime=0 streaming_engine.js:1431
(video:1) startup complete streaming_engine.js:1963
(all) setting up Period 0 streaming_engine.js:975
(all) Period 0 is being or has been set up streaming_engine.js:969
(all) Stream 1 is being or has been set up streaming_engine.js:1043
canSwitch_ player.js:3090
Jumping forward 3.0459998304431792 seconds to catch up with the seek range. playhead.js:258
(all): seeked: buffered seek: playheadTime=92841.73726677895 streaming_engine.js:792
Load latency: 7.53 player.js:939
Updating manifest... hls_parser.js:2142
First segment 27-05959.ts starts at 92845.75826666669 hls_parser.js:1403
Jumping forward 5.886266643530689 seconds to catch up with the seek range. playhead.js:258
(all): seeked: buffered seek: playheadTime=92848.611266613 streaming_engine.js:792
Calling switch_(), bandwidth=50537 kbps simple_abr_manager.js:254
switch_ player.js:3149
switch: Stream (video:1) already active streaming_engine.js:734
Jumping forward 3.293266637803754 seconds to catch up with the seek range. playhead.js:258
(all): seeked: buffered seek: playheadTime=92853.4142665863 streaming_engine.js:792
Updating manifest... hls_parser.js:2142
Flushing media pipeline due to stall inside buffered range gap_jumping_controller.js:275
(all): seeked: buffered seek: playheadTime=92854.67026662827 streaming_engine.js:792
First segment 33-04961.ts starts at 92851.71726666669 hls_parser.js:1403
Jumping forward 3.314266693108948 seconds to catch up with the seek range. playhead.js:258
(all): seeked: buffered seek: playheadTime=92858.95826673508 streaming_engine.js:792
Updating manifest... hls_parser.js:2142
First segment 38-04960.ts starts at 92856.67826666668 hls_parser.js:1403
Flushing media pipeline due to stall inside buffered range gap_jumping_controller.js:275
(all): seeked: buffered seek: playheadTime=92860.2892665863 streaming_engine.js:792
Jumping forward 3.009266721724998 seconds to catch up with the seek range. playhead.js:258
(all): seeked: buffered seek: playheadTime=92862.0502667427 streaming_engine.js:792
Calling switch_(), bandwidth=40423 kbps simple_abr_manager.js:254
switch_ player.js:3149
switch: Stream (video:1) already active streaming_engine.js:734
Updating manifest...

[avelad]

I suspect that the problem is similar to #1704

[Kogoruhn]

@avelad looks so. Btw, I've tried your decision with looking for webos ua, but the problem didnt pass.

[Kogoruhn]

Perfomance comparing gave interesting results: 22ms against 1071 ms

PC

WebOS

[Kogoruhn]

Ive extracted transmuxing logic to worker, js-timings become cool. But big freezings still remained (lol)
Then I increased 'trust interval' to 2k and so now i have almost normal playback with tiny, hardly-seen freezes.

[vaage]

@Kogoruhn since you and @avelad both see this as similar to #1704 I am going to label in the same way. However like in #1704, I am not sure how much we will be able to do since we are not well set-up to test smart tvs.

In the meantime, I would encourage you to contribute to the conversation in the #1753 which is about exploring ways that we can all work together to better investigate SmartTV issues.

[avelad]

@vaage , Is it possible isolate transmuxing logic into worker (https://caniuse.com/#search=web%20workers) inside Shaka Player?

[avelad]

I just tested with link removed and the fix for #1820 and I don't see the stall in webos.

Still, I think it would be nice to move mux.js to a worker.

[Kogoruhn]

@avelad I think so too :)
Thanks for the fix, ill try it later!
And please, remove stream link from your last message, I deleted link from topic start just now.

[vaage]

@avelad, to my understanding, we can't use service works in the Player library.

We have one with the demo, but not in the library. My understanding (which could be out-of-date) is an app can only have one service worker, so if we were to add one in the library, it would conflict with the application.

What is the goal of moving transmuxing?

[avelad]

@vaage web workers are different than service workers. See https://bitsofco.de/web-workers-vs-service-workers-vs-worklets/

[vaage]

@avelad Thank you for providing the additional information, that looks very interesting and useful in more than one space. Adopting this sounds like it would be a shift in architecture. Decisions like this are above my level, @joeyparrish, what are your thoughts on this?

[Kogoruhn]

When i create my own worker, i have keeked in videojs transmuxer worker. If anybody want, i can attach result here.

[joeyparrish]

Since the fix for #1820 is working with the content here, this doesn't seem to be a unique bug.

But I think transmuxing in a worker thread would be great. I've renamed the issue to reflect the goal, and tagged it as "enhancement".