DASH + ad insertion: Uncaught exception in promise in 4.7.0

[fredrik-telia]

Have you read the FAQ and checked for duplicate open issues?
yes

If the problem is related to FairPlay, have you read the tutorial?

What version of Shaka Player are you using?

4.6.3, 4.7.1

Can you reproduce the issue with our latest release version?
yes

Can you reproduce the issue with the latest code from main?
n/a

Are you using the demo app or your own custom app?
custom

If custom app, can you reproduce the issue using our demo app?
yes

What browser and OS are you using?
Chrome/MacOS

For embedded devices (smart TVs, etc.), what model and firmware version are you using?

What are the manifest and license server URIs?

Can be provided in email if needed

What configuration are you using? What is the output of player.getConfiguration()?

Default demo app

What did you do?

Play dash manifest with ad insertion, problem occurs when switching between content and ads

What did you expect to happen?
No uncaught exception in the console

What actually happened?

Error in the console:

The codecs in the array:

The error says MANIFEST.HLS_COULD_NOT_GUESS_CODECS but HLS is not involved.

Content will continue playing, but the console will fill up with errors.

I didn't see the issue on 4.6.3, but I did see it in 4.7.0, perhaps this commit is a good place to start?
24e3255

[cristian-atehortua]

Thanks @avelad for the fix.

Theme apart don't you think the name of that error is wrong? I think it should be changed in the next major release and, in the meantime, fix the documentation to say that it's not only related to HLS.

[littlespex]

This issue still occurs, and the HLS error message is very misleading when it occurs during DASH playback.

[avelad]

@littlespex can you send a fix for it? Thanks!

[cristian-atehortua]

@littlespex can you share in which scenario and which Shaka's version is happening? For our case, the issue is no longer happening for version >=4.7.4.

[joeyparrish]

This change in v4.7.4 could explain an exception being fixed:

https://github.com/shaka-project/shaka-player/pull/6073/files#diff-4264ba5cebc3bca4b16e2beb0d42d2679f27d7a9ed3f43d57ab2dc17471a0e19L236-L241

[littlespex]

This issue happens in all 4.7.x versions for us. This is an issue related to period flattening:

The stream we are testing is a DAI generated manifest that has mixed video codecs across periods:

• Ads have avc1 and hev adaptations
• Content has dvhe, hvc1, and avc1 adaptations

We are using to preferrVideoCodec to favor non-avc1 codecs. Because there is no matching advanced codec from ads -> content, the period flattening logic combines hev and hvc1 into a single codec string, separated by a comma. Later, when it tries to guess the correct codec to play, its sees the comma and thinks its a video + audio codec string. It splits the string and assumes the second value is the audio codec, which it's not, and throws the ill named HLS_COULD_NOT_GUESS_CODECS because niether hvc or hev are valid audio codecs.

[willdharris]

I'm working with @littlespex and @dsparacio on this issue.

With our content, after the streams from all periods are combined, we end up with multiple video streams that include multiple codecs concatenated in the stream's codecs property. For example: hev1.2.4.L63.90,hvc1.2.4.L63.90 and dvhe.05.01,avc1.42c00d.

Later on when the manifest is filtered for decoding configs and media capabilities, we hit the issue @littlespex described above. If codecs includes a comma, it's assumed the stream is multiplexed (video + audio) and ultimately fails with HLS_COULD_NOT_GUESS_CODECS because the second codec is not a valid audio stream.

We identified this became an issue for us with the addition of the Handle mixed-codec variants feature: #5950, where codecs of the same content type are being concatenated.

shaka-player/lib/util/periods.js

Lines 786 to 795 in 7fd99b7

	if (used) {
	unusedStreamsPerPeriod[i].delete(match);
	// Add the codec of this stream to the output stream's codecs.
	const codecs = new Set(outputStream.codecs.split(','));
	for (const codec of match.codecs.split(',')) {
	codecs.add(codec);
	}
	outputStream.codecs = Array.from(codecs).join(',');
	}
	}

I'm not sure I fully understand the intention here. Does a stream need to list both codecs for the mixed-codec feature to work properly? Are these codecs combined to say, "if the first codec is not available in a period but the second is, use the second"?

But then with #6047, filterDuplicateCodecs_ was added which essentially removes the second concatenated codec.

shaka-player/lib/util/periods.js

Lines 725 to 760 in 4ae15c2

	if (used) {
	unusedStreamsPerPeriod[i].delete(match);
	// Add the codec of this stream to the output stream's codecs.
	const codecs = new Set(outputStream.codecs.split(','));
	for (const codec of match.codecs.split(',')) {
	if (codec) {
	codecs.add(codec);
	}
	}
	const PeriodCombiner = shaka.util.PeriodCombiner;
	outputStream.codecs = PeriodCombiner.filterDuplicateCodecs_(
	Array.from(codecs)).join(',');
	}
	}
	}
	/**
	* @param {!Array.<string>} codecs
	* @return {!Array.<string>} codecs
	* @private
	*/
	static filterDuplicateCodecs_(codecs) {
	// Filter out duplicate codecs.
	const seen = new Set();
	const ret = [];
	for (const codec of codecs) {
	const shortCodec = shaka.util.MimeUtils.getCodecBase(codec);
	if (!seen.has(shortCodec)) {
	ret.push(codec);
	seen.add(shortCodec);
	} else {
	shaka.log.debug('Ignoring duplicate codec');
	}
	}
	return ret;
	}

That fix works when concatenated codecs have the same codec base, i.e. avc and avc avc1.42c01e, avc1.640028, but doesn't work if the codec bases are different, i.e hev and hvc hev1.2.4.L63.90,hvc1.2.4.L63.90. Which probably explains why the originally reported issue was resolved with the fix but not with our content.

My outstanding questions regarding the approach we should take for a fix:

1. Should the concatenated codecs be filtered to remove the "duplicate" codec? We can add a fix here for the additional codecs that aren't being caught. But does this break the mixed-codec feature?
2. Should the codecs remain concatenated? Then in the stream utils where multiplexed streams are split, add a check that only runs the multiplexed code if the concatenated codecs are different types (video vs audio).

[dsparacio]

More context, our content carries hvc and dvhe. hev is inserted by DAI. Additionally, DAI is mixing color space when DVHE is selected as they do not plan to provide a DVHE adpod encode as of today.

In 4.3.6, the mixed case was working with HEV and HVC but not HEV and DVHE. So this is why we looked into latest and found this new issue. If we remove DAI from the mix, 4.7.11 is working as expected regarding the codec selection and period flattening. We hope if we fix the mix codecs issue we should be able to get the expected outcome for period mixtures

 [HEV] [DVHE] [HEV] [DVHE] [HEV]
 [HEV] [HVC] [HEV] [HVC] [HEV]

We are working with the DAI team, asking for DVHE to be available in addition to hevc for other reasons beyond this issue.

[theodab]

I'm not sure I fully understand the intention here. Does a stream need to list both codecs for the mixed-codec feature to work properly? Are these codecs combined to say, "if the first codec is not available in a period but the second is, use the second"?

This was a fix for an issue from an internal client, who had multiperiod streams where some periods only had opus audio and some periods only had aac audio.

Should the concatenated codecs be filtered to remove the "duplicate" codec? We can add a fix here for the additional codecs that aren't being caught. But does this break the mixed-codec feature?

I think it might break the mixed-codec feature, yeah. Annoyingly the sample content I was given for the original issue is no longer hosted online, so to confirm this I would have to make new content.

Should the codecs remain concatenated? Then in the stream utils where multiplexed streams are split, add a check that only runs the multiplexed code if the concatenated codecs are different types (video vs audio).

I think this would be safe? I'd feel happier if I could test it on some real content though.

Honestly, it'd probably be a good idea to make some content for reproducing the original issue that can be put into the Shaka Player demo anyway, so we can confirm that future changes don't break this feature. I'll hack something together tomorrow, and try those solutions out on it.

[willdharris]

@theodab Thanks for the response. Yeah I came across the code in getDecodingInfosForVariant_ where the concatenated codecs are each checked for support. So I see why both codecs should be there. The problem right now, is that with two video codecs, we don't reach that part of the code due to an earlier error in getDecodingConfigs_. I think it makes sense to add a check to only run the multiplexed code for multiplexed content.

I have a fix in progress and will try to get a PR up today.

[theodab]

I made a sample asset, and tested a basic solution along the lines of the second solution you mentioned.

I think there is just a fundamental problem in how I wrote #5950.
The sample content I had, each period had a different codec but the same mimeType.
And the solution I wrote handled that case, but it doesn't handle variants that have multiple mimeTypes properly. Specifically, if there are multiple mimeTypes, we will test compatibility only with the first mimeType. For example, if we go from video/webm; codecs="vp09.00.41.08" to video/mp4; codecs="avc1.4d401f", we test to see if the stream is supported as though it was video/webm; codecs="vp09.00.41.08,avc1.4d401f"... which is wrong, because at no point are we using avc1 in webm, a combination not supported by Chrome.

This is kind of a problem because we still might create these variants, even though we can't play them. Also because it means the documentation for manifest.dash.multiTypeVariantsAllowed is wrong.

I'll make a change to fix how the multi-type support handles streams that switch between mimeTypes. MediaSource can play such content, we just get confused when we try to determine if they are playable or not. I think this will involve reverting how codecs are stored in streams (so they won't be combined together anymore), and instead add a new value that just stores an array of the final full mimeTypes of each stream and uses that for compatibility checks.

[willdharris]

Thanks @theodab. There were still some issues with the fix I was working on last week, so I did not submit a PR. We'll defer to your changes and re-test when that's available.

[dsparacio]

@theodab any idea on your timeline on this change. Just to feedback to the DAI team next Monday when we meet on this.

[theodab]

I have an experimental PR out for this now. I've tested it with the sample asset I made, and it seems to help, but it'd be nice to know if it solves the actual content you are having issues with.

[willdharris]

Thanks @theodab. I tested our content with your branch theodab:fixMultiMimeTypeBranch. This gets us past the HLS_COULD_NOT_GUESS_CODECS error, but now we're seeing a 6001 / REQUESTED_KEY_SYSTEM_CONFIG_UNAVAILABLE error.

I'm also getting this error with other DRM content (not mixed codecs) in fixMultiMimeTypeBranch. That same content plays with no errors in main.

"Error: Shaka Error DRM.REQUESTED_KEY_SYSTEM_CONFIG_UNAVAILABLE ()
    at new shaka.util.Error (http://127.0.0.1:5501/lib/util/error.js:101:15)
    at shaka.media.DrmEngine.queryMediaKeys_ (http://127.0.0.1:5501/lib/media/drm_engine.js:955:13)
    at shaka.media.DrmEngine.init_ (http://127.0.0.1:5501/lib/media/drm_engine.js:412:20)
    at async shaka.media.PreloadManager.initializeDrmInner_ (http://127.0.0.1:5501/lib/media/preload_manager.js:538:5)
    at async shaka.media.PreloadManager.start (http://127.0.0.1:5501/lib/media/preload_manager.js:387:7)
    at async mutexWrapOperation (http://127.0.0.1:5501/lib/player.js:1407:9)
    at async shaka.Player.load (http://127.0.0.1:5501/lib/player.js:1449:11)"

We are working on getting shareable content that we can pass along.

[willdharris]

I think I found the cause of the REQUESTED_KEY_SYSTEM_CONFIG_UNAVAILABLE error. Added a comment on the PR.

[willdharris]

@theodab The last commit resolves the REQUESTED_KEY_SYSTEM_CONFIG_UNAVAILABLE error now. However we've hit one more issue with our content. This a DAI stream, with clear and encrypted periods. The pre-roll ad plays, but does not transition to the encrypted content period. No errors in the logs.

In this branch, I see 2 calls to MediaKeys.createSession both at the end of the ad period, just before the encrypted content period.
In previous Shaka versions where this content works (i.e. 4.6.15), there is a single call to MediaKeys.createSession at the very start of the pre-roll ad.

I also found there may be a conflict with the Remove DOM Parser feature. If I revert that commit, and related commits (#6063, #6198, #6267) from your branch the issue is resolved. Not sure of the correlation yet.

Still working on getting a shareable stream. Will send that over asap.

[willdharris]

@theodab I just emailed content URIs for our test content to the maintainers group.

Again, the initial HLS_COULD_NOT_GUESS_CODECS error is resolved with the current PR. Now the period transition from unencrypted pre-roll to encrypted content is not working. Reproducible on Desktop Chrome.

[willdharris]

Tested the fix again today with our content and confirmed the HLS_COULD_NOT_GUESS_CODECS issue is resolved.

The ad to content freeze appears to be due to the content period being Widevine L1 encrypted. There's no problem with playback on HW CDM devices. In previous versions of Shaka, on non-HW CDMs, the content falls back to SD variants and playback continues. Now playback just stalls with no error. But that is unrelated to this specific issue and we will open a separate issue if needed.

[theodab]

There we go. Sorry it took so long to merge that!