Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media Session API: Add artwork #162

Merged
merged 9 commits into from
Oct 5, 2021
+163 −10
Merged

Media Session API: Add artwork #162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
1f12b7d
Merge pull request #131 from xwp/develop
derekherman Jun 3, 2021
8aed526
Merge pull request #132 from xwp/develop
derekherman Jun 3, 2021
85ce961
Merge pull request #135 from xwp/develop
derekherman Jul 16, 2021
cb22e57
Merge pull request #141 from GoogleChrome/develop
derekherman Aug 6, 2021
e2eeaa2
Merge pull request #160 from GoogleChrome/develop
derekherman Sep 23, 2021
534a465
Media Session API: Provide artwork for all videos. Cache artwork.
dero Sep 29, 2021
d7ac820
Cache Media Session artwork on download and play.
dero Oct 4, 2021
64ca172
Avoid overwriting cache entries on every play or download.
dero Oct 4, 2021
ab8c104
Fill in default artwork when no video specific artwork is found.
dero Oct 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions src/api/basics/01-single-video.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,26 @@ video-sources:
- src: https://storage.googleapis.com/kino-assets/single-video/video.mp4
type: video/mp4; codecs="avc1.640032,mp4a.40.2"
thumbnail: https://storage.googleapis.com/kino-assets/single-video/thumbnail.png
media-session-artwork:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we already have linter warnings so that we don't forget to add media session artwork to future .md files?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see how that would be useful, but I don't believe I've ever linted front-matter blocks in markdown files. What tool would you recommend for the task?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://www.npmjs.com/package/remark-frontmatter maybe. We use it for web.dev. See https://github.com/GoogleChrome/web.dev/search?q=remark-frontmatter

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think adding a markdown linter is a great idea. However, it feels a bit out of scope for this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted. Do you mind filing a feature request to keep track of it?

- sizes: 96x96
src: https://storage.googleapis.com/kino-assets/single-video/artwork-96x96.png
type: image/png
- sizes: 128x128
src: https://storage.googleapis.com/kino-assets/single-video/artwork-128x128.png
type: image/png
- sizes: 192x192
src: https://storage.googleapis.com/kino-assets/single-video/artwork-192x192.png
type: image/png
- sizes: 256x256
src: https://storage.googleapis.com/kino-assets/single-video/artwork-256x256.png
type: image/png
- sizes: 384x384
src: https://storage.googleapis.com/kino-assets/single-video/artwork-384x384.png
type: image/png
- sizes: 512x512
src: https://storage.googleapis.com/kino-assets/single-video/artwork-512x512.png
type: image/png

---

## Introduction
Expand Down
19 changes: 19 additions & 0 deletions src/api/basics/02-multiple-sources.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,25 @@ video-sources:
- src: https://storage.googleapis.com/kino-assets/multiple-sources/vp9.webm
type: video/webm
thumbnail: https://storage.googleapis.com/kino-assets/multiple-sources/thumbnail.png
media-session-artwork:
- sizes: 96x96
src: https://storage.googleapis.com/kino-assets/multiple-sources/artwork-96x96.png
type: image/png
- sizes: 128x128
src: https://storage.googleapis.com/kino-assets/multiple-sources/artwork-128x128.png
type: image/png
- sizes: 192x192
src: https://storage.googleapis.com/kino-assets/multiple-sources/artwork-192x192.png
type: image/png
- sizes: 256x256
src: https://storage.googleapis.com/kino-assets/multiple-sources/artwork-256x256.png
type: image/png
- sizes: 384x384
src: https://storage.googleapis.com/kino-assets/multiple-sources/artwork-384x384.png
type: image/png
- sizes: 512x512
src: https://storage.googleapis.com/kino-assets/multiple-sources/artwork-512x512.png
type: image/png
---

## Introduction
Expand Down
19 changes: 19 additions & 0 deletions src/api/basics/03-using-webvtt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,25 @@ video-subtitles:
src: https://storage.googleapis.com/kino-assets/using-webvtt/cap-cz.vtt
srclang: cz
thumbnail: https://storage.googleapis.com/kino-assets/using-webvtt/thumbnail.png
media-session-artwork:
- sizes: 96x96
src: https://storage.googleapis.com/kino-assets/using-webvtt/artwork-96x96.png
type: image/png
- sizes: 128x128
src: https://storage.googleapis.com/kino-assets/using-webvtt/artwork-128x128.png
type: image/png
- sizes: 192x192
src: https://storage.googleapis.com/kino-assets/using-webvtt/artwork-192x192.png
type: image/png
- sizes: 256x256
src: https://storage.googleapis.com/kino-assets/using-webvtt/artwork-256x256.png
type: image/png
- sizes: 384x384
src: https://storage.googleapis.com/kino-assets/using-webvtt/artwork-384x384.png
type: image/png
- sizes: 512x512
src: https://storage.googleapis.com/kino-assets/using-webvtt/artwork-512x512.png
type: image/png
---

## Introduction
Expand Down
19 changes: 19 additions & 0 deletions src/api/streaming/04-streaming-basics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,25 @@ video-subtitles:
src: https://storage.googleapis.com/kino-assets/streaming-basics/cap-cz.vtt
srclang: cz
thumbnail: https://storage.googleapis.com/kino-assets/streaming-basics/thumbnail.png
media-session-artwork:
- sizes: 96x96
src: https://storage.googleapis.com/kino-assets/streaming-basics/artwork-96x96.png
type: image/png
- sizes: 128x128
src: https://storage.googleapis.com/kino-assets/streaming-basics/artwork-128x128.png
type: image/png
- sizes: 192x192
src: https://storage.googleapis.com/kino-assets/streaming-basics/artwork-192x192.png
type: image/png
- sizes: 256x256
src: https://storage.googleapis.com/kino-assets/streaming-basics/artwork-256x256.png
type: image/png
- sizes: 384x384
src: https://storage.googleapis.com/kino-assets/streaming-basics/artwork-384x384.png
type: image/png
- sizes: 512x512
src: https://storage.googleapis.com/kino-assets/streaming-basics/artwork-512x512.png
type: image/png
---

## Introduction
Expand Down
19 changes: 19 additions & 0 deletions src/api/streaming/05-efficient-formats.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,25 @@ video-subtitles:
src: https://storage.googleapis.com/kino-assets/efficient-formats/cap-cz.vtt
srclang: cz
thumbnail: https://storage.googleapis.com/kino-assets/efficient-formats/thumbnail.png
media-session-artwork:
- sizes: 96x96
src: https://storage.googleapis.com/kino-assets/efficient-formats/artwork-96x96.png
type: image/png
- sizes: 128x128
src: https://storage.googleapis.com/kino-assets/efficient-formats/artwork-128x128.png
type: image/png
- sizes: 192x192
src: https://storage.googleapis.com/kino-assets/efficient-formats/artwork-192x192.png
type: image/png
- sizes: 256x256
src: https://storage.googleapis.com/kino-assets/efficient-formats/artwork-256x256.png
type: image/png
- sizes: 384x384
src: https://storage.googleapis.com/kino-assets/efficient-formats/artwork-384x384.png
type: image/png
- sizes: 512x512
src: https://storage.googleapis.com/kino-assets/efficient-formats/artwork-512x512.png
type: image/png
---

## Introduction
Expand Down
19 changes: 19 additions & 0 deletions src/api/streaming/06-adaptive-streaming.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,25 @@ video-subtitles:
src: https://storage.googleapis.com/kino-assets/adaptive-streaming/cap-cz.vtt
srclang: cz
thumbnail: https://storage.googleapis.com/kino-assets/adaptive-streaming/thumbnail.png
media-session-artwork:
- sizes: 96x96
src: https://storage.googleapis.com/kino-assets/adaptive-streaming/artwork-96x96.png
type: image/png
- sizes: 128x128
src: https://storage.googleapis.com/kino-assets/adaptive-streaming/artwork-128x128.png
type: image/png
- sizes: 192x192
src: https://storage.googleapis.com/kino-assets/adaptive-streaming/artwork-192x192.png
type: image/png
- sizes: 256x256
src: https://storage.googleapis.com/kino-assets/adaptive-streaming/artwork-256x256.png
type: image/png
- sizes: 384x384
src: https://storage.googleapis.com/kino-assets/adaptive-streaming/artwork-384x384.png
type: image/png
- sizes: 512x512
src: https://storage.googleapis.com/kino-assets/adaptive-streaming/artwork-512x512.png
type: image/png
---

## Introduction
Expand Down
2 changes: 1 addition & 1 deletion src/js/pages/Video.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ export default (routerContext) => {
playButton.addEventListener('click', (e) => {
const videoContainer = e.target.closest('.video-container');
const playerWrapper = videoContainer.querySelector('.video-container--player');
const videoPlayer = new VideoPlayer();
const videoPlayer = new VideoPlayer(downloader);

videoContainer.classList.add('has-player');
videoPlayer.render(currentVideoData);
Expand Down
38 changes: 33 additions & 5 deletions src/js/web-components/video-download/VideoDownloader.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ import DownloadManager from '../../classes/DownloadManager';
import StorageManager from '../../classes/StorageManager';
import getURLsForDownload from '../../utils/getURLsForDownload';

import { MEDIA_SESSION_DEFAULT_ARTWORK } from '../../constants';

export default class VideoDownloader extends HTMLElement {
static get observedAttributes() {
return ['state', 'progress', 'downloading', 'willremove'];
Expand Down Expand Up @@ -180,6 +182,19 @@ export default class VideoDownloader extends HTMLElement {
return subtitlesUrls;
}

/**
* Returns the artwork images URLs used by Media Session API
* to render the media popup / notification.
*
* @returns {string[]} URLs.
*/
getMediaSessionArtworkUrls() {
const artworkObjects = this.internal.videoData['media-session-artwork'] || MEDIA_SESSION_DEFAULT_ARTWORK;
const artworkUrls = artworkObjects.map((artworkObject) => artworkObject.src);

return artworkUrls;
}

/**
* Saves assets to the specified cache using Cache API.
*
Expand All @@ -188,7 +203,12 @@ export default class VideoDownloader extends HTMLElement {
async saveToCache(urls) {
try {
const cache = await caches.open(this.internal.cacheName);
await cache.addAll(urls);

urls.forEach(async (url) => {
if (!await cache.match(url)) {
await cache.add(url);
}
});
} catch (error) {
if (error.name === 'QuotaExceededError') {
/**
Expand All @@ -203,14 +223,22 @@ export default class VideoDownloader extends HTMLElement {

/**
* Downloads the current video and its assets to the cache and IDB.
*
* @param {object} opts Download options.
* @param {boolean} opts.assetsOnly Whether to cache only video assets: poster images,
* subtitles and Media Session API artowrk.
*/
async download() {
async download(opts = {}) {
const posterURLs = this.getPosterURLs();
const subtitlesURLs = this.getSubtitlesUrls();
const mediaSessionArtworkURLs = this.getMediaSessionArtworkUrls();

this.saveToCache([...posterURLs, ...subtitlesURLs, ...mediaSessionArtworkURLs]);

this.downloading = true;
this.saveToCache([...posterURLs, ...subtitlesURLs]);
this.runIDBDownloads();
if (!opts.assetsOnly) {
this.downloading = true;
this.runIDBDownloads();
}
}

/**
Expand Down
18 changes: 14 additions & 4 deletions src/js/web-components/video-player/VideoPlayer.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,16 @@ import selectSource from '../../utils/selectSource';
import { MEDIA_SESSION_DEFAULT_ARTWORK } from '../../constants';

export default class extends HTMLElement {
constructor() {
/**
* @param {VideoDownloader} downloader Video downloader associated with the current video.
*/
constructor(downloader) {
super();

this.internal = {};
this.internal.root = this.attachShadow({ mode: 'open' });
this.internal = {
downloader,
root: this.attachShadow({ mode: 'open' }),
};
}

/**
Expand Down Expand Up @@ -159,7 +164,7 @@ export default class extends HTMLElement {
title: this.internal.videoData.title || '',
artist: this.internal.videoData.artist || '',
album: this.internal.videoData.album || '',
artwork: MEDIA_SESSION_DEFAULT_ARTWORK,
artwork: this.internal.videoData['media-session-artwork'] || MEDIA_SESSION_DEFAULT_ARTWORK,
});

/**
Expand Down Expand Up @@ -283,10 +288,15 @@ export default class extends HTMLElement {
*/
play() {
const HAVE_NOTHING = 0;

if (this.videoElement.readyState === HAVE_NOTHING) {
this.videoElement.addEventListener('loadeddata', this.videoElement.play, { once: true });
} else {
this.videoElement.play();
}

this.internal.downloader.download({
assetsOnly: true,
});
}
}