feat: add modern EME support for FairPlay (#3776)

Add support for HLS com.apple.streamingkeydelivery through MSE/EME implementation. Close #3346 ## Tests Tested on: - Mac 11.6 Safari 15.2 - iOS 15.2 Safari 15.2 - Mac 11.6 Chrome 96 (for potential regressions on Widevine keySystem) | Mode | DRM API | TS | CMAF (mono-key and multi-keys) |---|---|---|---| | file | EME | ✅ | ✅ | | file | Legacy-prefixed | ✅ | ✅ | | media-source | EME | **mux-js**: `encrypted` never fired<br />**real MSE**: `encrypted` event received, but with incorrect `sinf` initData (*1) | ✅ | | media-source | Legacy-prefixed | **mux-js**: `webkitneedkey` never fired<br/>**real MSE**: TBD | 🔴 fails to append media segment to SourceBuffer (init segment ok) `(video:4) – "failed fetch and append: code=3015"` | ## Support table | Mode | DRM API | TS | CMAF (mono-key and multi-keys) |---|---|---|---| | file | EME | ✅ | ✅ | | file | Legacy-prefixed | ✅ | ✅ | | media-source | EME | 🚫 `4040: HLS_MSE_ENCRYPTED_MP2T_NOT_SUPPORTED` | ✅ | | media-source | Legacy-prefixed | 🚫 `4041: HLS_MSE_ENCRYPTED_LEGACY_APPLE_MEDIA_KEYS_NOT_SUPPORTED` |🚫 `4041: HLS_MSE_ENCRYPTED_LEGACY_APPLE_MEDIA_KEYS_NOT_SUPPORTED` | ⚠️ Use EME APIs with multi-keys CMAF makes the video stalling with the audio continuing alone after a short time (~3 minutes in the stream, could be shorter, could be longer). Didn't find an explanation to that yet. I've observed the same behaviour with hls.js code so I don't think this is a player issue.
shaka-project · Feb 7, 2022 · 6d76a13 · 6d76a13
1 parent 7155ec7
commit 6d76a13
Show file tree

Hide file tree

Showing 16 changed files with 395 additions and 185 deletions.
diff --git a/README.md b/README.md
@@ -171,7 +171,9 @@ NOTES:
 |HLS       |**Y**     |**Y**    |**Y** ¹  | -        |
 
 NOTES:
- - ¹: We support FairPlay through Apple's native HLS player.
+ - ¹: By default, FairPlay is handled using Apple's native HLS player, when on
+   Safari. We do support FairPlay through MSE/EME, however. See the
+   `streaming.useNativeHlsOnSafari` configuration value.
 
 
 ## Media container and subtitle support

diff --git a/docs/tutorials/drm-config.md b/docs/tutorials/drm-config.md
@@ -184,6 +184,11 @@ Microsoft Documentation: https://docs.microsoft.com/en-us/playready/overview/sec
 
 NB: Audio Hardware DRM is not supported (PlayReady limitation)
 
+##### FairPlay
+
+Based on [Apple's Documentation](https://developer.apple.com/streaming/fps/),
+you should provide an empty string as robustness
+
 ##### Other key-systems
 
 Values for other key systems are not known to us at this time.

diff --git a/docs/tutorials/fairplay.md b/docs/tutorials/fairplay.md
@@ -1,25 +1,31 @@
 # FairPlay Support
 
-When using native `src=` playback, we support using FairPlay on Safari.
+We support FairPlay with EME on compatible environments or native `src=`.
 Adding FairPlay support involves a bit more work than other key systems.
 
-
 ## Server certificate
 
-All FairPlay content requires setting a server certificate.  This is set in the
-Player configuration:
+All FairPlay content requires setting a server certificate. You can either
+provide it directly or set a serverCertificateUri for Shaka to fetch it for
+you.
 
 ```js
 const req = await fetch('https://example.com/cert.der');
 const cert = await req.arrayBuffer();
 
-player.configure('drm.advanced.com\\.apple\\.fps\\.1_0.serverCertificate',
+player.configure('drm.advanced.com\\.apple\\.fps\\.serverCertificate',
                  new Uint8Array(cert));
 ```
 
+```js
+player.configure('drm.advanced.com\\.apple\\.fps\\.serverCertificateUri',
+                 'https://example.com/cert.der');
+```
 
 ## Content ID
 
+Note: This only applies when legacy Apple Media Keys is used.
+
 Some FairPlay content use custom signaling for the content ID.  The content ID
 is used by the browser to generate the license request.  If you don't use the
 default content ID derivation, you need to specify a custom init data transform:

diff --git a/externs/polyfill.js b/externs/polyfill.js
@@ -0,0 +1,15 @@
+/*! @license
+ * Shaka Player
+ * Copyright 2016 Google LLC
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+/**
+ * @fileoverview Externs for Shaka polyfills
+ *
+ * @externs
+ */
+
+
+/** @type {boolean} */
+window.shakaMediaKeysPolyfill;
diff --git a/externs/shaka/player.js b/externs/shaka/player.js
@@ -934,8 +934,9 @@ shaka.extern.ManifestConfiguration;
  * @property {boolean} useNativeHlsOnSafari
  *   Desktop Safari has both MediaSource and their native HLS implementation.
  *   Depending on the application's needs, it may prefer one over the other.
- *   Examples: FairPlay is only supported via Safari's native HLS, but it
- *   doesn't have an API for selecting specific tracks.
+ *   Warning when disabled: Where single-key DRM streams work fine, multi-keys
+ *   streams is showing unexpected behaviours (stall, audio playing with video
+ *   freezes, ...). Use with care.
  * @property {number} inaccurateManifestTolerance
  *   The maximum difference, in seconds, between the times in the manifest and
  *   the times in the segments.  Larger values allow us to compensate for more

diff --git a/lib/hls/hls_parser.js b/lib/hls/hls_parser.js
@@ -40,6 +40,7 @@ goog.require('shaka.util.Networking');
 goog.require('shaka.util.OperationManager');
 goog.require('shaka.util.Pssh');
 goog.require('shaka.util.Timer');
+goog.require('shaka.util.Platform');
 goog.requireType('shaka.hls.Segment');
 
 
@@ -1391,6 +1392,21 @@ shaka.hls.HlsParser = class {
           shaka.util.Error.Code.HLS_INVALID_PLAYLIST_HIERARCHY);
     }
 
+    /** @type {!Array.<!shaka.hls.Tag>} */
+    const variablesTags = shaka.hls.Utils.filterTagsByName(playlist.tags,
+        'EXT-X-DEFINE');
+
+    const mediaVariables = this.parseMediaVariables_(variablesTags);
+
+    goog.asserts.assert(playlist.segments != null,
+        'Media playlist should have segments!');
+
+    this.determinePresentationType_(playlist);
+
+    /** @type {string} */
+    const mimeType = await this.guessMimeType_(type, codecs, playlist,
+        mediaVariables);
+
     /** @type {!Array.<!shaka.hls.Tag>} */
     const drmTags = [];
     if (playlist.segments) {
@@ -1425,7 +1441,7 @@ shaka.hls.HlsParser = class {
         const drmParser =
             shaka.hls.HlsParser.KEYFORMATS_TO_DRM_PARSERS_[keyFormat];
 
-        const drmInfo = drmParser ? drmParser(drmTag) : null;
+        const drmInfo = drmParser ? drmParser(drmTag, mimeType) : null;
         if (drmInfo) {
           if (drmInfo.keyIds) {
             for (const keyId of drmInfo.keyIds) {
@@ -1446,21 +1462,6 @@ shaka.hls.HlsParser = class {
           shaka.util.Error.Code.HLS_KEYFORMATS_NOT_SUPPORTED);
     }
 
-    /** @type {!Array.<!shaka.hls.Tag>} */
-    const variablesTags = shaka.hls.Utils.filterTagsByName(playlist.tags,
-        'EXT-X-DEFINE');
-
-    const mediaVariables = this.parseMediaVariables_(variablesTags);
-
-    goog.asserts.assert(playlist.segments != null,
-        'Media playlist should have segments!');
-
-    this.determinePresentationType_(playlist);
-
-    /** @type {string} */
-    const mimeType = await this.guessMimeType_(type, codecs, playlist,
-        mediaVariables);
-
     // MediaSource expects no codec strings combined with raw formats.
     // TODO(#2337): Instead, create a Stream flag indicating a raw format.
     if (shaka.hls.HlsParser.RAW_FORMATS_.includes(mimeType)) {
@@ -2807,6 +2808,41 @@ shaka.hls.HlsParser = class {
     return op.promise;
   }
 
+  /**
+   * @param {!shaka.hls.Tag} drmTag
+   * @param {string} mimeType
+   * @return {?shaka.extern.DrmInfo}
+   * @private
+   */
+  static fairplayDrmParser_(drmTag, mimeType) {
+    if (mimeType == 'video/mp2t') {
+      throw new shaka.util.Error(
+          shaka.util.Error.Severity.CRITICAL,
+          shaka.util.Error.Category.MANIFEST,
+          shaka.util.Error.Code.HLS_MSE_ENCRYPTED_MP2T_NOT_SUPPORTED);
+    }
+
+    if (shaka.util.Platform.isMediaKeysPolyfilled()) {
+      throw new shaka.util.Error(
+          shaka.util.Error.Severity.CRITICAL,
+          shaka.util.Error.Category.MANIFEST,
+          shaka.util.Error.Code
+              .HLS_MSE_ENCRYPTED_LEGACY_APPLE_MEDIA_KEYS_NOT_SUPPORTED);
+    }
+
+    /*
+     * Even if we're not able to construct initData through the HLS tag, adding
+     * a DRMInfo will allow DRM Engine to request a media key system access
+     * with the correct keySystem and initDataType
+     */
+    const drmInfo = shaka.util.ManifestParserUtils.createDrmInfo(
+        'com.apple.fps', [
+          {initDataType: 'sinf', initData: new Uint8Array(0)},
+        ]);
+
+    return drmInfo;
+  }
+
   /**
    * @param {!shaka.hls.Tag} drmTag
    * @return {?shaka.extern.DrmInfo}
@@ -3029,7 +3065,7 @@ shaka.hls.HlsParser.EXTENSION_MAP_BY_CONTENT_TYPE_ = {
 
 
 /**
- * @typedef {function(!shaka.hls.Tag):?shaka.extern.DrmInfo}
+ * @typedef {function(!shaka.hls.Tag, string):?shaka.extern.DrmInfo}
  * @private
  */
 shaka.hls.HlsParser.DrmParser_;
@@ -3040,10 +3076,8 @@ shaka.hls.HlsParser.DrmParser_;
  * @private
  */
 shaka.hls.HlsParser.KEYFORMATS_TO_DRM_PARSERS_ = {
-  /* TODO: https://github.com/google/shaka-player/issues/382
   'com.apple.streamingkeydelivery':
       shaka.hls.HlsParser.fairplayDrmParser_,
-  */
   'urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed':
       shaka.hls.HlsParser.widevineDrmParser_,
   'com.microsoft.playready':