Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cast): Use cast platform APIs in MediaCapabilties polyfill #4727

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
aa973af
Initial fix with temp debug logs.
JulianDomingo Nov 17, 2022
0ee3554
Removed console debug logs after verifying eotf param on a 4K HDR Rok…
JulianDomingo Nov 18, 2022
4873f27
Updated comment on smpte2048 check.
JulianDomingo Nov 18, 2022
2d56c99
Merge branch 'main' into 4726-account-for-extended-mime-type-paramete…
JulianDomingo Nov 19, 2022
ef09c30
Added existence check for transferFunction parameter.
JulianDomingo Nov 19, 2022
51a96ea
Handled review comments (iter #0) and added unit tests. Also correcte…
JulianDomingo Nov 19, 2022
9588678
Fixed unit test linting errors.
JulianDomingo Nov 19, 2022
b60d59a
Restored cast namespace after polyfill test executions.
JulianDomingo Nov 19, 2022
bf81981
Use window.cast instead of cast for namespace checking.
JulianDomingo Nov 19, 2022
279b029
Sanity check to confirm issue with CDT() support on Selenium lab test…
JulianDomingo Nov 22, 2022
1dd0722
Updated canDisplayType() checks to guard against instable implementat…
JulianDomingo Nov 22, 2022
6a8cf51
Fall back to MediaSource.isTypeSupported() if cast APIs aren't availa…
JulianDomingo Nov 22, 2022
d0701ea
Removed window.cast to trigger CAST_API_UNAVAILABLE error for lab chr…
JulianDomingo Nov 23, 2022
720fe11
Verifying lab tests fail without check for cast.__platform__ and cast…
JulianDomingo Nov 23, 2022
8465a99
Re-instated deleted code from previous commit, which prevented cast l…
JulianDomingo Nov 23, 2022
be1065a
Handling PR review comments.
JulianDomingo Nov 29, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 59 additions & 9 deletions lib/polyfill/media_capabilities.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ goog.provide('shaka.polyfill.MediaCapabilities');

goog.require('shaka.log');
goog.require('shaka.polyfill');
goog.require('shaka.util.Error');
goog.require('shaka.util.Platform');


Expand Down Expand Up @@ -96,37 +97,48 @@ shaka.polyfill.MediaCapabilities = class {
return res;
}

const videoConfig = mediaDecodingConfig['video'];
const audioConfig = mediaDecodingConfig['audio'];

if (mediaDecodingConfig.type == 'media-source') {
if (!shaka.util.Platform.supportsMediaSource()) {
return res;
}
// Use 'MediaSource.isTypeSupported' to check if the stream is supported.
if (mediaDecodingConfig['video']) {
const contentType = mediaDecodingConfig['video'].contentType;
const isSupported = MediaSource.isTypeSupported(contentType);
// Cast platforms will additionally check canDisplayType(), which
// accepts extended MIME type parameters.
// See: https://github.com/shaka-project/shaka-player/issues/4726
if (videoConfig) {
let isSupported;
if (shaka.util.Platform.isChromecast()) {
isSupported =
shaka.polyfill.MediaCapabilities.canCastDisplayType_(videoConfig);
} else {
isSupported = MediaSource.isTypeSupported(videoConfig.contentType);
}
if (!isSupported) {
return res;
}
}

if (mediaDecodingConfig['audio']) {
const contentType = mediaDecodingConfig['audio'].contentType;
if (audioConfig) {
const contentType = audioConfig.contentType;
const isSupported = MediaSource.isTypeSupported(contentType);
if (!isSupported) {
return res;
}
}
} else if (mediaDecodingConfig.type == 'file') {
if (mediaDecodingConfig['video']) {
const contentType = mediaDecodingConfig['video'].contentType;
if (videoConfig) {
const contentType = videoConfig.contentType;
const isSupported = shaka.util.Platform.supportsMediaType(contentType);
if (!isSupported) {
return res;
}
}

if (mediaDecodingConfig['audio']) {
const contentType = mediaDecodingConfig['audio'].contentType;
if (audioConfig) {
const contentType = audioConfig.contentType;
const isSupported = shaka.util.Platform.supportsMediaType(contentType);
if (!isSupported) {
return res;
Expand Down Expand Up @@ -217,6 +229,44 @@ shaka.polyfill.MediaCapabilities = class {
return res;
}

/**
* Checks if the given media parameters of the video or audio streams are
* supported by the Cast platform.
* @param {!VideoConfiguration} videoConfig The 'video' field of the
* MediaDecodingConfiguration.
* @return {boolean} `true` when the stream can be displayed on a Cast device.
* @private
*/
static canCastDisplayType_(videoConfig) {
if (!(window.cast)) {
shaka.log.error('Expected cast namespace to be available!');
throw new shaka.util.Error(
shaka.util.Error.Severity.CRITICAL,
shaka.util.Error.Category.CAST,
shaka.util.Error.Code.CAST_API_UNAVAILABLE);
} else if (!(cast.__platform__ && cast.__platform__.canDisplayType)) {
shaka.log.warning('Expected cast APIs to be available! Falling back to ' +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very strange. I'd like to help you look into that, but I don't want to hold up the PR for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, was held up on this for a while - the lab tests were failing without this check.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent a solid day on this, and it's time for me to give up for now. I managed to forward the Cast platform APIs through an iframe boundary, but not in a way that is compatible with Karma's websocket usage. :-(

'MediaSource.isTypeSupported() for type support.');
return MediaSource.isTypeSupported(videoConfig.contentType);
}

let displayType = videoConfig.contentType;
if (videoConfig.width && videoConfig.height) {
displayType +=
`; width=${videoConfig.width}; height=${videoConfig.height}`;
}
if (videoConfig.framerate) {
displayType += `; framerate=${videoConfig.framerate}`;
}
if (videoConfig.transferFunction === 'pq') {
// A "PQ" transfer function indicates this is an HDR-capable stream;
// "smpte2084" is the published standard. We need to inform the platform
// this query is specifically for HDR.
displayType += '; eotf=smpte2084';
}
return cast.__platform__.canDisplayType(displayType);
}

/**
* A method for generating a key for the MediaKeySystemAccessRequests cache.
*
Expand Down
103 changes: 103 additions & 0 deletions test/polyfill/media_capabilities_unit.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,18 @@
*/

describe('MediaCapabilities', () => {
const Util = shaka.test.Util;
const originalCast = window['cast'];
const originalVendor = navigator.vendor;
const originalUserAgent = navigator.userAgent;
const originalRequestMediaKeySystemAccess =
navigator.requestMediaKeySystemAccess;
const originalMediaCapabilities = navigator.mediaCapabilities;

/** @type {MediaDecodingConfiguration} */
let mockDecodingConfig;
/** @type {!jasmine.Spy} */
let mockCanDisplayType;

beforeAll(() => {
Object.defineProperty(window['navigator'],
Expand Down Expand Up @@ -64,9 +69,17 @@ describe('MediaCapabilities', () => {
},
};
shaka.polyfill.MediaCapabilities.memoizedMediaKeySystemAccessRequests_ = {};

mockCanDisplayType = jasmine.createSpy('canDisplayType');
mockCanDisplayType.and.returnValue(false);
});

afterEach(() => {
window['cast'] = originalCast;
});

afterAll(() => {
window['cast'] = originalCast;
Object.defineProperty(window['navigator'],
'userAgent', {value: originalUserAgent});
Object.defineProperty(window['navigator'],
Expand Down Expand Up @@ -172,5 +185,95 @@ describe('MediaCapabilities', () => {
expect(requestKeySystemAccessSpy)
.toHaveBeenCalledTimes(1);
});

it('throws when the cast namespace is not available', async () => {
// Temporarily remove window.cast to trigger error. It's restored after
// every test.
delete window['cast'];

const isChromecastSpy =
spyOn(shaka['util']['Platform'],
'isChromecast').and.returnValue(true);
JulianDomingo marked this conversation as resolved.
Show resolved Hide resolved
const expected = Util.jasmineError(new shaka.util.Error(
shaka.util.Error.Severity.CRITICAL,
shaka.util.Error.Category.CAST,
shaka.util.Error.Code.CAST_API_UNAVAILABLE));
const isTypeSupportedSpy =
spyOn(window['MediaSource'], 'isTypeSupported').and.returnValue(true);

shaka.polyfill.MediaCapabilities.install();
await expectAsync(
navigator.mediaCapabilities.decodingInfo(mockDecodingConfig))
.toBeRejectedWith(expected);

expect(isTypeSupportedSpy).not.toHaveBeenCalled();
// 1 (during install()) + 1 (for video config check).
expect(isChromecastSpy).toHaveBeenCalledTimes(2);
});

it('falls back to isTypeSupported() when canDisplayType() missing',
async () => {
// We only set the cast namespace, but not the canDisplayType() API.
window['cast'] = {};
const isChromecastSpy =
spyOn(shaka['util']['Platform'],
'isChromecast').and.returnValue(true);
const isTypeSupportedSpy =
spyOn(window['MediaSource'], 'isTypeSupported')
.and.returnValue(true);

shaka.polyfill.MediaCapabilities.install();
await navigator.mediaCapabilities.decodingInfo(mockDecodingConfig);

expect(mockCanDisplayType).not.toHaveBeenCalled();
// 1 (during install()) + 1 (for video config check).
expect(isChromecastSpy).toHaveBeenCalledTimes(2);
// 1 (fallback in canCastDisplayType()) +
// 1 (mockDecodingConfig.audio).
expect(isTypeSupportedSpy).toHaveBeenCalledTimes(2);
});

it('should use cast.__platform__.canDisplayType for "supported" field ' +
'when platform is Cast', async () => {
// We're using quotes to access window.cast because the compiler
// knows about lots of Cast-specific APIs we aren't mocking. We
// don't need this mock strictly type-checked.
window['cast'] = {
__platform__: {canDisplayType: mockCanDisplayType},
};
const isChromecastSpy =
spyOn(shaka['util']['Platform'],
'isChromecast').and.returnValue(true);
const isTypeSupportedSpy =
spyOn(window['MediaSource'], 'isTypeSupported').and.returnValue(true);

// Tests an HDR stream's extended MIME type is correctly provided.
mockDecodingConfig.video.transferFunction = 'pq';
mockDecodingConfig.video.contentType =
'video/mp4; codecs="hev1.2.4.L153.B0"';
// Round to a whole number since we can't rely on number => string
// conversion precision on all devices.
mockDecodingConfig.video.framerate = 24;
mockCanDisplayType.and.callFake((type) => {
expect(type).toBe(
'video/mp4; ' +
'codecs="hev1.2.4.L153.B0"; ' +
'width=512; ' +
'height=288; ' +
'framerate=24; ' +
'eotf=smpte2084');
return true;
});

shaka.polyfill.MediaCapabilities.install();
await navigator.mediaCapabilities.decodingInfo(mockDecodingConfig);

// 1 (during install()) + 1 (for video config check).
expect(isChromecastSpy).toHaveBeenCalledTimes(2);
// 1 (mockDecodingConfig.audio).
expect(isTypeSupportedSpy).toHaveBeenCalledTimes(1);
// Called once in canCastDisplayType.
expect(mockCanDisplayType).toHaveBeenCalledTimes(1);
});
});
});