Skip to content

Commit

Permalink
Bug 1566603 [wpt PR 17416] - MSE: Test addSourceBuffer and changeType…
Browse files Browse the repository at this point in the history 
… relaxed type strictness and allowance for implicit changeType, a=testonly

Automatic update from web-platform-tests
MSE: Test addSourceBuffer and changeType relaxed type strictness and allowance for implicit changeType

In preparation for Chrome's relaxation of addSourceBuffer and
changeType codec specificity within the mime type parameter, this
change adds new new tests. It also updates the Blink test expectations
for these new tests to be failures. Later changes will add Chrome's
implementation of the relaxed strictness, and will also remove the
failure expectations.

external/wpt/media-source/mediasource-changetype-play-without-codecs-parameter:
  Tests successful addSourceBuffer, changeType, and codec switching
  without using any codecs parameter in types passed to addSourceBuffer
  and changeType for pairs of test media that are of same media class
  (audio or video) and are single track. This test is very similar to
  mediasource-changetype-play.html, just with less specific parameters
  to addSourceBuffer and changeType. This file is kept separate from
  mediasource-changetype-play.html to help identify implementations
  (like Chrome at the time of this change) that require more specific
  parameters to those methods.

external/wpt/media-source/mediasource-changetype-play-implicit.html:
  Tests successful codec switching without using changeType for test
  media of the same bytestream format, separately for audio-only and
  video-only pairs of media. Also includes a set of the same tests where
  the initial addSourceBuffer uses only the (relaxed) mime type/subtype
  without any codecs parameters, for any pairs that included codecs
  parameters in their full types.
  Note: In Chrome, only 1 pair of actually different media files is
  supported across the test media: webm vp8 and vp9. We can add more
  test media later if greater coverage is desired.

external/wpt/media-source/mediasource-changetype-play-negative.html:
  Tests of various explicit, implicit, strict and relaxed scenarios
  which should fail.

Main MSE spec issue for this:
  w3c/media-source#161
Related codec-switching MSE spec issue:
  w3c/media-source#155
These new tests exercise related scenarios to the new non-normative
  guidance in the codec-switching incubation spec branch in WICG
  (https://github.com/WICG/media-source/tree/codec-switching),
  clarifying that implementations can be relaxed in addSourceBuffer
  and changeType codec-specificity, but not isTypeSupported; apps
  should still provide as much type specifics as they can to achieve
  earlier confidence in support or lack thereof, and to avoid issues
  with implementations (like Chrome at the time of this change) that
  require more specific parameters to those methods.

BUG=535738

Change-Id: I20fd477b2429ef94ee70bf57ed8c18543774da93
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1663349
Reviewed-by: Dan Sanders <sandersdchromium.org>
Commit-Queue: Matthew Wolenetz <wolenetzchromium.org>
Cr-Commit-Position: refs/heads/master{#672316}

--
Fix async tests in media-source

mediasource-util.js redefines test.done() to add a final assertion
before actually calling done(). However, the assertion was not wrapped
in test.step(), which would lead to uncaught errors.

--

wpt-commits: b42e788c6937bce1317411e61e9d18ba06835d82, 012de5bb12d78ad7135c26b15fc24f366b1c5845
wpt-pr: 17416

UltraBlame original commit: 87512e5f2ef881420cf09f14c96520b4281cccb0
  • Loading branch information
@marco-c
marco-c committed Oct 4, 2019
1 parent 3f09887 commit a16b968
Show file tree
Hide file tree
Showing 6 changed files with 423 additions and 21 deletions.
89 changes: 89 additions & 0 deletions testing/web-platform/tests/media-source/mediasource-changetype-play-implicit.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
<!DOCTYPE html>

<html>
<head>
<title>Exercise implicit changeType for supported test types, using mime types WITH and WITHOUT codecs for addSourceBuffer.</title>
<meta name="timeout" content="long">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="mediasource-util.js"></script>
<script src="mediasource-changetype-util.js"></script>
</head>
<body>
<div id="log"></div>
<script>

// Helper that generates implicit codec switching tests for a pair of media
// types, with full codecs in the original addSourceBuffer calls, and
// separately without full codecs parameters in the original addSourceBuffer
// calls.
function generateTestsForMediaPair(type1, type2) {
// Implicit changeType across bytestream formats is not expected to be
// supported, so skip those tests' generation.
if (type1.mime_subtype != type2.mime_subtype)
return;

assert_equals(type1.is_video, type2.is_video,
"Types must both be audio or both be video");
test_description_prefix = "Test " + (type1.is_video ? "video" : "audio") +
"-only implicit changeType for " + type1.type + " <-> " + type2.type;

mediaSourceChangeTypeTest(
type1, type2,
test_description_prefix,
{ implicit_changetype: true } );

// Skip test generation if the relaxed types are already fully specified and
// tested, above.
if (type1.type == type1.relaxed_type &&
type2.type == type2.relaxed_type) {
return;
}

mediaSourceChangeTypeTest(
type1, type2,
test_description_prefix +
" (using types without codecs parameters for addSourceBuffer)",
{ use_relaxed_mime_types: true, implicit_changetype: true } );
}

function generateImplicitChangeTypeTests(audio_types, video_types) {
async_test((test) => {
// Require at least 1 pair of different audio-only or video-only test media
// files sharing same bytestream format.
assert_true(audio_types.length > 1 || video_types.length > 1,
"Browser doesn't support enough test media");

// Count the number of unique bytestream formats used in each of audio_types
// and video_types.
let audio_formats = new Set(Array.from(audio_types, t => t.mime_subtype));
let video_formats = new Set(Array.from(video_types, t => t.mime_subtype));
assert_true(audio_types.length > audio_formats.size ||
video_types.length > video_formats.size,
"Browser doesn't support at least 2 audio-only or 2 video-only test " +
"media with same bytestream format");

test.done();
}, "Check if browser supports enough test media types and pairs of " +
"audio-only or video-only media with same bytestream format");

// Generate audio-only tests
for (let audio1 of audio_types) {
for (let audio2 of audio_types) {
generateTestsForMediaPair(audio1, audio2);
}
}

// Generate video-only tests
for (let video1 of video_types) {
for (let video2 of video_types) {
generateTestsForMediaPair(video1, video2);
}
}
}

findSupportedChangeTypeTestTypes(generateImplicitChangeTypeTests);

</script>
</body>
</html>
122 changes: 122 additions & 0 deletions testing/web-platform/tests/media-source/mediasource-changetype-play-negative.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
<!DOCTYPE html>

<html>
<head>
<title>Exercise scenarios expected to fail for changeType for supported test types, using mime types WITH and WITHOUT codecs.</title>
<meta name="timeout" content="long">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="mediasource-util.js"></script>
<script src="mediasource-changetype-util.js"></script>
</head>
<body>
<div id="log"></div>
<script>

function generateNegativeChangeTypeTests(audio_types, video_types) {
async_test((test) => {
assert_true(audio_types.length > 0 && video_types.length > 0,
"Browser doesn't support at least one audio and one video test media for audio<->video changeType negative tests");

let audio_formats = new Set(Array.from(audio_types, t => t.mime_subtype));
let video_formats = new Set(Array.from(video_types, t => t.mime_subtype));

let has_intersected_av_format = false;
for (let elem of audio_formats) {
if (video_formats.has(elem))
has_intersected_av_format = true;
}
assert_true(has_intersected_av_format,
"Browser doesn't support at least 1 audio-only and 1 video-only test media with same bytestream formats");

test.done();
}, "Check if browser supports enough test media types across audio and video for changeType negative tests");

// Generate audio<->video changeType tests that should not succeed in
// reaching successful end of playback because the class of media (audio or
// video) must remain the same across either an implicit or explicit
// changeType.
for (let audio_type of audio_types) {
for (let video_type of video_types) {
// For implicit changeType negative tests, only pairs of test media files
// using the same bytestream format are used, because it is not
// guaranteed that all implementations can be expected to reliably detect
// an implicit switch of bytestream format (for example, MP3 parsers
// might skip invalid input bytes without issuing error.)
let do_implicit_changetype = (audio_type.mime_subtype ==
video_type.mime_subtype);

mediaSourceChangeTypeTest(
audio_type, video_type,
"Negative test audio<->video changeType for " +
audio_type.type + " <-> " + video_type.type,
{ negative_test: true } );
mediaSourceChangeTypeTest(
video_type, audio_type,
"Negative test video<->audio changeType for " +
video_type.type + " <-> " + audio_type.type,
{ negative_test: true } );

if (do_implicit_changetype) {
mediaSourceChangeTypeTest(
audio_type, video_type,
"Negative test audio<->video implicit changeType for " +
audio_type.type + " <-> " + video_type.type,
{ implicit_changetype: true, negative_test: true } );
mediaSourceChangeTypeTest(
video_type, audio_type,
"Negative test video<->audio implicit changeType for " +
video_type.type + " <-> " + audio_type.type,
{ implicit_changetype: true, negative_test: true } );
}

// Skip tests where the relaxed type is already fully specified and
// tested, above.
if (audio_type.type == audio_type.relaxed_type &&
video_type.type == video_type.relaxed_type) {
continue;
}

mediaSourceChangeTypeTest(
audio_type, video_type,
"Negative test audio<->video changeType for " +
audio_type.type + " <-> " + video_type.type +
" (using types without codecs parameters)",
{ use_relaxed_mime_types: true, negative_test: true } );
mediaSourceChangeTypeTest(
video_type, audio_type,
"Negative test video<->audio changeType for " +
video_type.type + " <-> " + audio_type.type +
" (using types without codecs parameters)",
{ use_relaxed_mime_types: true, negative_test: true } );

if (do_implicit_changetype) {
mediaSourceChangeTypeTest(
audio_type, video_type,
"Negative test audio<->video implicit changeType for " +
audio_type.type + " <-> " + video_type.type +
" (without codecs parameters for addSourceBuffer)",
{ use_relaxed_mime_types: true,
implicit_changetype: true,
negative_test: true
} );

mediaSourceChangeTypeTest(
video_type, audio_type,
"Negative test video<->audio implicit changeType for " +
video_type.type + " <-> " + audio_type.type +
" (without codecs parameters for addSourceBuffer)",
{ use_relaxed_mime_types: true,
implicit_changetype: true,
negative_test: true
} );
}
}
}
}

findSupportedChangeTypeTestTypes(generateNegativeChangeTypeTests);

</script>
</body>
</html>
52 changes: 52 additions & 0 deletions ...web-platform/tests/media-source/mediasource-changetype-play-without-codecs-parameter.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
<!DOCTYPE html>

<html>
<head>
<title>Exercise changeType for supported test types, using mime types WITHOUT codecs for addSourceBuffer and changeType.</title>
<meta name="timeout" content="long">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="mediasource-util.js"></script>
<script src="mediasource-changetype-util.js"></script>
</head>
<body>
<div id="log"></div>
<script>

function generateChangeTypeTests(audio_types, video_types) {
async_test((test) => {
assert_true(audio_types.length > 1, "Browser doesn't support at least 2 types of audio test media");
assert_true(video_types.length > 1, "Browser doesn't support at least 2 types of video test media");
test.done();
}, "Check if browser supports enough test media types");

// Generate audio-only changeType tests
for (let audio1 of audio_types) {
for (let audio2 of audio_types) {
mediaSourceChangeTypeTest(
audio1, audio2,
"Test audio-only changeType for " +
audio1.type + " <-> " + audio2.type +
" (using types without codecs parameters)",
{ use_relaxed_mime_types: true } );
}
}

// Generate video-only changeType tests
for (let video1 of video_types) {
for (let video2 of video_types) {
mediaSourceChangeTypeTest(
video1, video2,
"Test video-only changeType for " +
video1.type + " <-> " + video2.type +
" (using types without codecs parameters)",
{ use_relaxed_mime_types: true });
}
}
}

findSupportedChangeTypeTestTypes(generateChangeTypeTests);

</script>
</body>
</html>
12 changes: 9 additions & 3 deletions testing/web-platform/tests/media-source/mediasource-changetype-play.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<html>
<head>
<title>Exercise changeType for supported test types.</title>
<title>Exercise changeType for supported test types, using mime types WITH codecs (if applicable) for addSourceBuffer and changeType.</title>
<meta name="timeout" content="long">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
Expand All @@ -23,14 +23,20 @@
// Generate audio-only changeType tests
for (let audio1 of audio_types) {
for (let audio2 of audio_types) {
mediaSourceChangeTypeTest(audio1, audio2, "Test audio-only changeType for " + audio1.type + " <-> " + audio2.type);
mediaSourceChangeTypeTest(
audio1, audio2,
"Test audio-only changeType for " +
audio1.type + " <-> " + audio2.type);
}
}

// Generate video-only changeType tests
for (let video1 of video_types) {
for (let video2 of video_types) {
mediaSourceChangeTypeTest(video1, video2, "Test video-only changeType for " + video1.type + " <-> " + video2.type);
mediaSourceChangeTypeTest(
video1, video2,
"Test video-only changeType for " +
video1.type + " <-> " + video2.type);
}
}
}
Expand Down
Loading

0 comments on commit a16b968

Please sign in to comment.