Standardization of 360 video playback

[SamiraAtMicrosoft]

Introduction

There currently exists no standard for immersive presentation of spherical video content on the web. As a result, web developers are required to ramp up on a steep learning curve involving WebGL (for mapping the video textures to the expected projection type such as cubemap or equirectangular) and WebXR (for displaying the content to an AR/VR device) in addition to the standard media API's. Other options include integrating a third-party engine that might alleviate this work, however developers of sites that do not otherwise require a rendering or game engine are reluctant to introduce the extra code required.

With this, there also exists a need for standardizing subtitle support in an AR/VR domain (issues 39 and 40) which is an important component of meeting accessibility requirements and should benefit all users (for instance in cases when a user is in a quiet environment and does not have headphones) thus increasing engagement.

Standardization options:

Options for where 360 video support can be brought to web developers can include:

• as an attribute on the video element
• part of the underlying spherical metadata that exists in the video container

The immersive web community has made significant progress in the WebXR standard which can be leveraged in the internal implementation.

Action items

I've opened this issue as a forum to discuss how 360 video can be standardized on the web. Content creators, service providers and other interested parties are also encouraged to provide feedback on their requirements for facilitating their content to become available in VR.

Final thoughts

Spherical video content is a prominent use case for VR devices. A standardized, native implementation provides benefits to web developers including ease of integration into their apps, likely performance gains and protection for DRM content.

[andreastai]

There has been a very good discussion at TPAC 2019 where @SamiraAtMicrosoft proposed and chaired a breakout session on this topic (see minutes).

There have been active participation by browser and immersive web people (e.g. @cwilso, @toji, @thetuvix, @MozillaReality, @jernoble).

I think it would be good to promote this issue somewhere in a repository and to start working on it. Because of the topic it could be a repo of the immersive web community group. If this does not fit, a separate community group on this topic may be an alternative.

[SamiraAtMicrosoft]

It looks like Mozilla has already built a custom element that supports 360 video:
https://github.com/MozillaReality/immersive-custom-elements

This could potentially be a good starting point to build upon where we can test out or integrate requested features such as subtitles.

[louaybassbouss]

Thanks @SamiraAtMicrosoft I like your proposal. We (at Fraunhofer FOKUS) mentioned a similar idea during the W3C Workshop on Web & Virtual Reality in 2016 (https://www.w3.org/2016/06/vr-workshop/slides/louay-360-video.pdf Slides 9 and 10). We will be happy to support you.