diff --git a/README.md b/README.md index e43436956..8e0a33df3 100644 --- a/README.md +++ b/README.md @@ -165,7 +165,7 @@ You can deploy a single HLS stream, code against the regular HTML5 video APIs, and create a fast, high-quality video experience across all the big web device categories. -Check out the [full documentation](docs/) for details on how HLS works +Check out the [full documentation](docs/intro.md) for details on how HLS works and advanced configuration. A description of the [adaptive switching behavior](docs/bitrate-switching.md) is available, too. diff --git a/docs/arch.md b/docs/arch.md index 804f76723..7fd0a23c8 100644 --- a/docs/arch.md +++ b/docs/arch.md @@ -1,22 +1,3 @@ -# HLS Overview -The HLS project polyfills support for [HTTP Live Streaming](https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/Introduction/Introduction.html) (HLS) video format. This document is intended as a primer for anyone interested in contributing or just better understanding how bits from a server get turned into video on their display. - -## HTTP Live Streaming -HLS has two primary characteristics that distinguish it from other video formats: - -- Delivered over HTTP(S): it uses the standard application protocol of the web to deliver all its data -- Segmented: longer videos are broken up into smaller chunks which can be downloaded independently and switched between at runtime - -A standard HLS stream consists of a *Master Playlist* which references one or more *Media Playlists*. Each Media Playlist contains references one or more sequential video segments. All these components form a logical hierarchy that informs the player of the different quality levels of the video available and how to address the individual segments of video at each of those levels: - -![HLS Format](hls-format.png) - -HLS streams can be delivered in two different modes: a "static" mode for videos that can be played back from any point, often referred to as video-on-demand (VOD); or a "live" mode where later portions of the video become available as time goes by. In the static mode, the Master and Media playlists are fixed. The player is guaranteed that the set of video segments referenced by those playlists will not change over time. - -Live mode can work in one of two ways. For truly live events, the most common configuration is for each individual Media Playlist to only include the latest video segment and a small number of consecutive previous segments. In this mode, the player may be able to seek backwards a short time in the video but probably not all the way back to the beginning. In the other live configuration, new video segments can be appended to the Media Playlists but older segments are never removed. This configuration allows the player to seek back to the beginning of the stream at any time during the broadcast and transitions seamlessly to the static stream type when the event finishes. - -If you're interested in a more in-depth treatment of the HLS format, check out [Apple's documentation](https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/Introduction/Introduction.html) and the IETF [Draft Specification](https://datatracker.ietf.org/doc/draft-pantos-http-live-streaming/). - ## HLS Project Overview This project has three primary duties: diff --git a/docs/glossary.md b/docs/glossary.md new file mode 100644 index 000000000..c1683754e --- /dev/null +++ b/docs/glossary.md @@ -0,0 +1,21 @@ +# Glossary + +**Playlist**: This is a representation of an HLS or DASH manifest. + +**Media Playlist**: This is a manifest that represents a single rendition of the source. + +**Master Playlist Controller**: This acts as the main controller for the playback engine. It interacts with the SegmentLoaders, PlaylistLoaders, PlaybackWatcher, etc. + +**Playlist Loader**: This will request the source and load the master manifest. It is also instructed by the ABR algorithm to load a media playlist or wraps a media playlist if it is provided as the source. There are more details about the playlist loader [here](./arch.md). + +**Segment Loader**: This determines which segment should be loaded, requests it via the Media Segment Loader and passes the result to the Source Updater. + +**Media Segment Loader**: This requests a given segment, decrypts the segment if necessary, and returns it to the Segment Loader. + +**Source Updater**: This manages the browser's [SourceBuffers](https://developer.mozilla.org/en-US/docs/Web/API/SourceBuffer). It appends decrypted segment bytes provided by the Segment Loader to the corresponding Source Buffer. + +**ABR(Adaptive Bitrate) Algorithm**: This concept is described more in detail [here](https://en.wikipedia.org/wiki/Adaptive_bitrate_streaming). Our chosen ABR algorithm is referenced by [selectPlaylist](../README.md#hlsselectplaylist) and is described more [here](./bitrate-switching.md). + +**Playback Watcher**: This attemps to resolve common playback stalls caused by improper seeking, gaps in content and browser issues. + +**Sync Controller**: This will attempt to create a mapping between the segment index and a display time on the player. diff --git a/docs/intro.md b/docs/intro.md new file mode 100644 index 000000000..f3ae4333c --- /dev/null +++ b/docs/intro.md @@ -0,0 +1,49 @@ +# Overview +This project supports both [HLS][hls] and [MPEG-DASH][dash] playback in the video.js player. This document is intended as a primer for anyone interested in contributing or just better understanding how bits from a server get turned into video on their display. + +## HTTP Live Streaming +[HLS][apple-hls-intro] has two primary characteristics that distinguish it from other video formats: + +- Delivered over HTTP(S): it uses the standard application protocol of the web to deliver all its data +- Segmented: longer videos are broken up into smaller chunks which can be downloaded independently and switched between at runtime + +A standard HLS stream consists of a *Master Playlist* which references one or more *Media Playlists*. Each Media Playlist contains one or more sequential video segments. All these components form a logical hierarchy that informs the player of the different quality levels of the video available and how to address the individual segments of video at each of those levels: + +![HLS Format](hls-format.png) + +HLS streams can be delivered in two different modes: a "static" mode for videos that can be played back from any point, often referred to as video-on-demand (VOD); or a "live" mode where later portions of the video become available as time goes by. In the static mode, the Master and Media playlists are fixed. The player is guaranteed that the set of video segments referenced by those playlists will not change over time. + +Live mode can work in one of two ways. For truly live events, the most common configuration is for each individual Media Playlist to only include the latest video segment and a small number of consecutive previous segments. In this mode, the player may be able to seek backwards a short time in the video but probably not all the way back to the beginning. In the other live configuration, new video segments can be appended to the Media Playlists but older segments are never removed. This configuration allows the player to seek back to the beginning of the stream at any time during the broadcast and transitions seamlessly to the static stream type when the event finishes. + +If you're interested in a more in-depth treatment of the HLS format, check out [Apple's documentation][apple-hls-intro] and the IETF [Draft Specification][hls-spec]. + +## Dynamic Adaptive Streaming over HTTP +Similar to HLS, [DASH][dash-wiki] content is segmented and is delivered over HTTP(s). + +A DASH stream consits of a *Media Presentation Description*(MPD) that describes segment metadata such as timing information, URLs, resolution and bitrate. Each segment can contain either ISO base media file format(e.g MP4) or MPEG-2 TS data. Typically, the MPD will describe the various *Representations* that map to collections of segments at different bitrates to allow bitrate selection. These Representations can be organized as a SegmentList, SegmentTemplate, SegmentBase, or SegmentTimeline. + +DASH streams can be delivered in both video-on-demand(VOD) and live streaming modes. In the VOD case, the MPD describes all the segments and representations available and the player can chose which representation to play based on it's capabilities. + +Live mode is accomplished using the ISOBMFF Live profile if the segments are in ISOBMFF. There are a few different ways to setup the MPD including but not limited to updating the MPD after an interval of time, using *Periods*, or using the *availabilityTimeOffset* field. A few examples of this are provided by the [DASH Reference Client][dash-if-reference-client]. The MPD will provide enough information for the player to playback the live stream and seek back as far as is specified in the MPD. + +If you're interested in a more in-depth description of MPEG-DASH, check out [MDN's tutorial on setting up DASH][mdn-dash-tut] or the [DASHIF Guidelines][dash-if-guide]. + +# Further Documentation + +- [Architechture](arch.md) +- [Glossary](glossary.md) +- [Adaptive Bitrate Switching](bitrate-switching.md) +- [Multiple Alternative Audio Tracks](multiple-alternative-audio-tracks.md) + +# Helpful Tools +- [FFmpeg](http://trac.ffmpeg.org/wiki/CompilationGuide) +- [Thumbcoil](http://thumb.co.il/): web based video inspector + +[hls]: /docs/intro.md#http-live-streaming +[dash]: /docs/intro.md#dynamic-adaptive-streaming-over-http +[apple-hls-intro]: https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/Introduction/Introduction.html +[hls-spec]: https://datatracker.ietf.org/doc/draft-pantos-http-live-streaming/ +[dash-wiki]: https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP +[dash-if-reference-client]: https://reference.dashif.org/dash.js/ +[mdn-dash-tut]: https://developer.mozilla.org/en-US/Apps/Fundamentals/Audio_and_video_delivery/Setting_up_adaptive_streaming_media_sources +[dash-if-guide]: http://dashif.org/guidelines/ \ No newline at end of file diff --git a/docs/live.md b/docs/live.md deleted file mode 100644 index b3c33593e..000000000 --- a/docs/live.md +++ /dev/null @@ -1,77 +0,0 @@ -# Video.JS HLS Live - -## Brightcove Service Differences: -- Brightcove uses the Zencoder HLS Live API. -- Once ingest begins, approximately 50 seconds later the original manifest is available. -- I have not seen any ability to enforce a sliding window, so I believe Brightcove Live HLS serves all available segments on a continuously growing manifest. -- One minute after last disconnect from the ingest stream, the event is considered complete and the final manifest delivered. -- The final manifest will be different in two distinct ways, it will include the `EXT-X-ENDLIST` tag notifying all connected clients that the live stream has concluded and is now VOD. It will also contain a custom `ZEN-TOTAL-DURATION`: tag with representing the total amount of recorded time in seconds. - -## Akamai Service Differences: -- Akamai only serves HLS Live off of Akamai HD2 endpoints. -- These vary from their HDS counterparts by url syntax. - - ``` /i/``` vs.``` /z/``` for HDS - - `master.m3u8` vs.`manifest.f4m` for HDS -- Their endpoints are difficult to arrange CORS configurations on. -- Akamai manifests span the gamut of known HLS tags, both supported and unsupported by our plugin. - -## Once Service Differences: -- Once manifests tend to include the use of `EXT-X-DISCONTINUITY` tags which are unsupported to date. -- Once streams so far tend to use a different encoding algorithm on their segments which sometime result in a range error during transmuxing. - - -# Live HLS Research -This document is a collection of notes on Live HLS implementations in the wild. - -There are two varieties of Live HLS. In the first, playlists are -persistent and strictly appended to. In the alternative form, the -maximum number of segments in a playlist is relatively stable and an -old segment is removed every time a new segment becomes available. - -On iOS devices, both stream types report a duration of `Infinity`. The -`currentTime` is equal to the amount of the stream that has been -played back on the device. - -## Akamai HD2 - -## OnceLIVE -"Sliding window" live streams. - -### Variant Playlists -Once variant playlists look like standard HLS variant playlists. - -### Media Playlists -OnceLIVE uses "sliding window" manifests for live playback. The media -playlists do not have an `EXT-X-ENDLIST` and don't declare a -`EXT-X-PLAYLIST-TYPE`. On first request, the stream media playlist -returned four segment URLs with a starting media sequence of one, -preceded by a `EXT-X-DISCONTINUITY` tag. As playback progressed, that -number grew to 13 segment URLs, at which point it stabilized. That -would equate to a steady-state 65 second window at 5 seconds per -segment. - -OnceLive documentation is [available on the Unicorn Media -website](http://www.unicornmedia.com/documents/2013/02/oncelive_implementationguide.pdf). - -Here's a script to quickly parse out segment URLs: - -```shell -curl $ONCE_MEDIA_PLAYLIST | grep '^http' -``` - -An example media playlist might look something like this: -```m3u8 -#EXTM3U -#EXT-X-TARGETDURATION:5 -#EXT-X-MEDIA-SEQUENCE:3 -#EXTINF:5,3 -http://example.com/0/1/content.ts?visitguid=uuid&asseturl=http://once.example.com/asset.lrm&failoverurl=http://example.com/blank.jpg -#EXTINF:5,4 -http://example.com/1/2/content.ts?visitguid=uuid&asseturl=http://once.example.com/asset.lrm&failoverurl=http://example.com/blank.jpg -#EXTINF:5,5 -http://example.com/2/3/content.ts?visitguid=uuid&asseturl=http://once.example.com/asset.lrm&failoverurl=http://example.com/blank.jpg -#EXTINF:5,6 -http://example.com/3/4/content.ts?visitguid=uuid&asseturl=http://once.example.com/asset.lrm&failoverurl=http://example.com/blank.jpg -``` - -## Zencoder Live