-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Semantic way to declare a video as an image #976
Comments
We already have the right semantic - |
The main concern with this is that we might need to expose a few events and some APIs (like |
So add a superclass and move some methods to there? |
I like the muted idea. |
Although that will still not meet the needs of web apps like vine.co, for which sound autoplay is a crucial part of the experience. Presumably this will force them into hacks involving using web audio to autoplay their sound. |
Hanging this behavior on
I think |
Alternatively, you could use an image format everyone forgot about a week after it was introduced: WebP. Of course, it's currently only supported on Chrome and Opera, but it IS supported on Chrome and Opera, unlike the current proposals here. |
I think we should treat autoplay with sound as a separate concern, it's going to be a lot more challenging to get that behavior in all browsers than this smaller of step of allowing autoplay if muted. The simplest thing one could do here, I think, is to allow autoplay if there is no audio track, and have a new attribute that causes audio tracks to be ignored. I wouldn't be possible to unmute. However, that fails to handle media resources where there is an audio track, but playing it muted is still meaningful. This is now fairly common in ads and I see this behavior on Twitter too I think. Something around the Edit: To be clear, the setter would still have to do nothing, |
I'd like to push back on this a little bit. In the end it's up to implementers what they're willing to do, but I think we shouldn't sink too much time into overdesigning this feature in service of a use case which is unclear. To me, the end goal should be to allow the So it's not clear to me what the use case of all the design work you outline is. It doesn't give authors any new capabilities that they can't already get through hacks, and it doesn't make things significantly easier to use, since the hacks are necessary if you want sound anyway. I guess you could argue that we want to make it easier for authors to play videos without sound easily, but I have serious doubts this is something any authors are asking for. The (in my estimation) very small number of authors for which this is appropriate can just use hacks, and don't need new design work, spec surface area, and implementation work done for them. It's also noteworthy that Firefox mobile already autoplays videos (including audio) by default. |
You are assuming that all mobile browsers allow autoplay for Web Audio which isn't true for Safari iOS. It wouldn't be surprising if other browsers like Chrome Android were to add the same kind of restrictions.
Firefox Android has a flag to disable autoplay which is bound to a UI toggle. It is really surprising that autoplay isn't disabled by default for them. Hopefully someone at Mozilla can give some context. |
I support looped autoplaying of video files with no audio tracks in
The last point is a bit of a risk/downside, because it'll mean that 1-frame WebM files with VP9/VP10 will be the best still image codec we have available on the web. |
That's a good point. But my point about it not being worth investing the feature/spec/implementation work on this, given that very few authors want it, still remains to be addressed. Is there data to the contrary, e.g. a large partner that would like video autoplay despite it having no sound? |
Interestingly, I just found this from Google: |
I'm entirely unconvinced by WebP vs WebM FAQ (it ignores cost of bandwidth wasted by WebP, and puts too much weight on Chrome-specific implementation problems) edited to shorten offtopicing rant |
This seems off-topic? |
@pornel - "Chrome-specific implementation problems" are important considering this is tagged with "needs implementor interest". |
Paging @ojanvafai. I think this would mostly be ads, videos as background images and Twitter-style videos where playback starts when you scroll by, but is muted by default. This happens on Twitter's Android app as well, so it's not a desktop-centric design choice. |
@domenic, I'd also like the autoplay restrictions to be more lax than they are now, but see no reason to think that audio playback without user interaction will become universally allowed any time soon to make this a non-problem. At the very least you'd have off-by-default settings for disabling autoplaying audio and things like do-not-disturb modes on phones. |
OK, so, I don't want to be obstructionist here, if I'm in the minority. Let's reset this conversation, with me just logging an objection that I really think we should allow autoplay of sound as well, and understanding that I'm overruled :). I'd like to understand exactly which use cases we're serving here, as I think it changes the shape of the solution a bit. In particular:
I guess there's nothing to say we can't do both. (2) would be useful for sites like Twitter, Facebook, or Vine, I imagine, whereas (1) would be useful for animated-gif replacement (ads, background images, transcoded gifs). For example, Twitter transcodes gifs, but also allows posting videos: maybe they would use (1) for uploaded transcoded gifs, and (2) for uploaded videos. I think doing both would be compelling mainly if browsers had ideas on how to use the additional semantic information: as you said, in things like media session or in different controls or menus. Anyway, as @mounirlamouri said, it would be good to get input from other vendors on this. Does anyone know the relevant people to ping? |
@padenot for Gecko, @jernoble @eric-carlson for WebKit FWIW, "autoplay-by-default experience, while requiring a user gesture to un-mute" (2) is what I'm hoping for, but I have no data, I've just seen that behavior in the wild. |
Also, I don't see this as an issue of semantics, rather just a matter of enabling the behaviors we want to enable. If there happens to be a semantic interpretation that's fine, of course. |
I think my main question is whether we can try to catch two birds with one stone and do (1) and (2) with the same solution. (1) can be see as a subset of (2) if switching from video-as-image to regular video is possible and allow then the video to be played unmuted. @foolip when you say that we should allow muted video to autoplay, do you mean |
My use case is a CDN that uses HTTP content negotiation (or UA sniffing) to automagically transcode GIF to WebM to offer 10× reduction in bandwidth (conversion to WebP or APNG is not worth the hassle). That will be possible only when |
Overall, I really like this idea. Perhaps because Mobile Safari is the most restrictive browser when it comes to automatic playback of media, we get a lot of requests to loosen our restrictions. They fall broadly into three sets of use cases:
@domenic asked "[is there any] large partner that would like video autoplay despite it having no sound?" Yes, each of these use cases have very large clients requesting support for these features in Mobile Safari. Each of these use cases would be solved with a different proposal which has been listed above. (1) (GIF replacement) would be solved by simply implementing support for .mp4 in an image element. (2) would be solved by (1), with the addition of simple playback controls to image elements. (Note that those additions would be useful even in the absence of (1), as it would allow page control of .gifs as well as .mp4s or .webms.) (3) could only be solved by a video element, with all the trappings and API that provides. |
@foolip said:
Rather than add a user-gesture requirement to the muted property itself (you have to do the same for VideoTrack.enabled, e.g.), it seems simpler to just stop automatic playback (i.e., |
@domenic said:
I think Media Session support is going to be very important. Authors using a video as a dynamic page element are probably not going to want, e.g., their videos to automatically play to an Apple TV. |
OK, so it sounds like the idea of allowing autoplay while muted and doing something special when trying to unmute without a user gesture is worth exploring more. Candidates for something are doing nothing or pausing. When doing nothing, there's the risk that the page has no button for the user to unmute and thus the video just plays silently, while if pausing there's the risk that there's no way for the user to resume playback, if it was intended that the video be controlled only by scripts. @jernoble, is your main concern with "do nothing" that it would break existing content, or that it's confusing to users or developers? Either solution seems simple enough implementation-wise I think. Would it help at all to measure how often scripts try to change muted from false to true with and without a user gesture? |
I agree that pausing might be better than doing nothing. Though, I think it would help to measure. We are also considering experimenting with this behaviour. |
Is there anything we could measure to help decide between pausing or doing nothing? Both seem kind of surprising to me, but I do lean towards doing nothing because at least it doesn't change another bit of state than was poked at. |
I can't think of anything that will help us. We could add a lot of metrics to find out if play/pause was called from a user gesture to find out if pausing would actually break but I don't think it will help given that it would mostly be wild guesses (UI could be removed, the element could be re-used). Finding whether the situation is common sounds more important given that it being uncommon means that we can focus on what's the best solution instead of backward compatibility. Note that I prefer pausing because it's something that should be smoother. Not un-muting when |
I think it the value should remain unchanged, and one can just check the attribute's value. (This is the same way that one can currently tell that a |
I think adding a |
I agree, it would be somewhat silly, the main point being feature detection really. I think this is the feature detection one would use otherwise, which seems fine: var v = document.createElement('video');
v.muted = true;
v.play();
var supportsGesturelessPlayWhileMuted = !v.paused; Let's see if I understand both options fully. Option 1 Option 2 I do like the symmetry of option 1, and it seems like a plus that it leaves the element in a state that we know is tolerable, because it was already in that state. |
I prefer Option 2. Sites should already be responsive to |
Well, OK, let's go with that then. Just to make sure, the hoped-for behavior is exactly what I wrote, with no extra state bit tracking if it has previously played audibly, has been previously unmuted by a user gesture, etc? The rules for |
Based on Intent to Implement and Ship: autoplay muted videos on Android on blink-dev, I think the necessary spec changes here are:
I'll assign to @mounirlamouri to write a PR or figure out who should :) |
I've started a thread in WICG/interventions: Regarding |
Is there a good way to feature detect autoplay inline-video in Android? Been digging around and I can't find anything. Right now i'm just using the UA string but that's is sort of unreliable. |
@milesthedisch, do you mean to detect whether function autoplayMutedSupported() {
var v = document.createElement('video');
v.muted = true;
v.play();
return !v.paused;
} (A media element doesn't actually need a src to try playing. It's all pretty weird.) |
I'm going to close this as this has been solved AFAICT. |
@foolip I am trying to detect if autoplay un-muted is possible, so I modified your function a bit by just removing function autoplayUnmutedSupported() {
var v = document.createElement('video');
v.play();
return !v.paused;
} Strangely enough I get Any ideas on why the unexpected result on Chrome 57 for Android? |
@alejandroiglesias Are you trying this in Chrome Dev Tools by any chance? Using DevTools might be recognized by Chrome for Android as a user gesture (for ease of debugging I guess) - that would explain the different result. https://jsfiddle.net/v09tq5sv/ shows false for me on Chrome for Android if opened without opening DevTools (from chrome://inspect works fine). |
@avayvod yes, using Chrome Devtools. Thanks a lot for chiming in and adding that info. As you say, it works fine on the fiddle in Chrome for Android. It's possible that running the function by hitting the |
Videos are often used as animated images and design elements on the Web on Desktop. On Mobile, websites will provide GIF or use
<canvas>
instead because of autoplay restrictions. This is wasting more bandwidth for the users who need it the most and use more energy on devices able to hardware decode the most popular video formats.I was wondering if it would make sense to have a declarative way to mark a video as an image so it can be used for GIF and design elements (like Uber, Apple or NYT do on their websites). Having an attribute for this would help feature detection and allow the UA to have different behaviour like the
controls
UI or the Media Session integration. Some browsers like Safari on iPhone could use the information to know that the video can be played inline for example.An alternative to this would be to make
<video autoplay muted>
play automatically in mobile browsers and blockmuted
change unless it has a user gesture. It has a few downsides: not semantically defined and slightly magic behaviour but has the benefit of being simple.Chrome would be interested to push something to the spec, whether a new attribute or make a different
muted
behaviour allowed by the spec. It would be great to hear from other vendors on this :)The text was updated successfully, but these errors were encountered: