Add sizes=auto to lazy-loaded <img>

[zcorpan]

This allows web developers to omit the sizes attribute, or explicitly specify sizes=auto, for <img> elements that have loading=lazy. The browser will use the layout width of the image as input to the srcset resource selection (lazy images don't start loading until layout is known, at the earliest). For eager-loading images, it's invalid and equivalent to 100vw.

Fixes #4654.

• At least two implementers are interested (and none opposed):
  • Chromium: yes Add "auto sizes" for lazy-loaded images #4654 (comment)
  • Gecko: yes Auto-sizes for lazy-loaded images in HTML mozilla/standards-positions#650
  • WebKit: yes Auto-sizes for lazy-loaded images in HTML WebKit/standards-positions#9
• Tests are written and can be reviewed and commented upon at:
  • HTML: test <img sizes=auto> web-platform-tests/wpt#34427
• Implementation bugs are filed:
  • Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1359051
  • Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1819581
  • Safari: https://bugs.webkit.org/show_bug.cgi?id=253143
  • Deno (only for timers, structured clone, base64 utils, channel messaging, module resolution, web workers, and web storage): N/A
  • Node.js (only for timers, structured clone, base64 utils, channel messaging, and module resolution): N/A
  • Validator.nu: Support <img sizes=auto> and <source sizes=auto> validator/validator#1396

(See WHATWG Working Mode: Changes for more details.)

---

/embedded-content.html ( diff )
/images.html ( diff )
/infrastructure.html ( diff )
/input.html ( diff )
/semantics.html ( diff )

[yoavweiss]

Thanks for working on this! :) A few nits + a substantive question on the processing model

[yoavweiss]

Is it worthwhile to integrate the concept of being rendered to a visual medium into the "being rendered" definition? As is, this is a bit confusing to read.

[zcorpan]

Hmm, I think the "is being rendered to a visual medium" is actually bogus. The width and height attributes used that, but I believe the branching should actually happen for "being rendered". Changed in f1d5577.

[yoavweiss]

What if we have a situation where the loading=lazy attribute was removed after the initial parsing of the <img> element? It's not a relevant mutation, but I would still expect sizes=auto to give us the layout width of the image, assuming that layout already happened.

[yoavweiss]

Might be better to ignore "auto" if layout hasn't happened yet, but take it into account if it has, regardless of the loading attribute

[zcorpan]

Taking "auto" into account if layout has happened makes it non-deterministic for eager images, as it's racy whether layout has happened before the image fetch starts. I wrote this PR with the assumption that it's better to have deterministic and predictable behavior. Which means that for eager images, auto = 100vw always.

Do we want non-deterministic here?

Or should we add a load() method for images, to explicitly start loading an image (like for media elements)?

[yoavweiss]

You're right that deterministic behavior would be better, and there's no simple way to enforce that without the loading attribute. I'm not sure there's a strong use-case for "this image was initially lazy, but now we need to load it eagerly", so maybe we can skip this for now, and look into explicit ways to eagerly load images in case it's needed.

[zcorpan]

I think loading needs to be a relevant mutation attribute, too, since it can change what sizes=auto means.

[zcorpan]

Added more conditions to relevant mutations.

With the rendered width being a possible relevant mutation, it could trigger on every frame (e.g. if the image dimensions are being animated). Is that a problem?

It sort of interferes with "Reacting to environment changes", since that aborts in 15.1 if there were relevant mutations (but maybe that's what we want for auto).

"update the image data" step 6.3.7.3 will fire a load event even if it's the same url, which doesn't seem great. If the image is broken, it will try to refetch it every time (#4480), which also doesn't seem great.

[yoavweiss]

Looks great! One question

[yoavweiss]

LGTM

[zcorpan]

I wonder if the feature should more explicitly check for the case of no dimensions being specified, since the behavior is a bit weird in that case.

• When no image is loaded yet, the width will be 0px, in which case the sizes algorithm returns 100vw.
• When an image is loaded, and a relevant mutation occurs (for example, the viewport width changes, making the rendered width of the image change because the used sizes was 100vw, and this is now a relevant mutation). This time, the parsed sizes will return the current rendered width of the image, in pixels. So it's no longer responsive to further changes of the viewport width.

So, how to fix this? Use 100vw if the computed value (not used value) of the 'width' and 'min-width' properties are both auto?

[yoavweiss]

So, how to fix this? Use 100vw if the computed value (not used value) of the 'width' and 'min-width' properties are both auto?

Using 100vw when we don't have any better alternatives (since we don't know what the layout size would be), SGTM. The "computed-value width isauto" case sounds like it fits that description. I'm not sure if there aren't other cases (but you probably know).

[zcorpan]

So, how to fix this? Use 100vw if the computed value (not used value) of the 'width' and 'min-width' properties are both auto?

Done in fc651c8.

I suppose since the 'aspect-ratio' property now exists, it's possible to specify a width by setting a height and an aspect-ratio. So if width and min-width are both auto, we should check if 'aspect-ratio' and 'height'/'min-height' are also auto.

Are there any other ways to specify a width? cc @whatwg/css

[zcorpan]

To answer my own question, I think we have these:

• 'inline-size' and 'block-size'; which is the width depends on 'writing-mode'
• sizing values (css-sizing-3 and css-sizing-4) can also be 'none', 'min-content', 'max-content', 'fit-content(...)', 'stretch', 'fit-content', 'contain', which in some cases act like 'auto'. Maybe we should check if the computed value is a <length-percentage> instead of checking that it's not 'auto'.
  • var() is substituted at computed-value time, so should work ok
  • not sure what to make of math functions
• as far as I can tell, per CSS2, for replaced elements (which img is before the image has loaded), the 'left' and 'right' properties can't be used to specify a width.

So, an img element has a "specified width" if at least one of the following is true:

• the computed value of 'width' or 'min-width' is a <length-percentage>
• the computed value of 'aspect-ratio' is not 'auto' and the computed value of 'height' or 'min-height' is a <length-percentage>
• the computed value of 'writing-mode' is 'horizontal-tb' and
  • the computed value of 'inline-size' or 'min-inline-size' is a <lenght-percentage> or
  • the computed value of 'aspect-ratio' is not 'auto' and is not degenerate and the computed value of 'block-size' or 'min-block-size' is a <length-percentage>
• the computed value of 'writing-mode' is not 'horizontal-tb' and
  • the computed value of 'block-size' or 'min-block-size' is a <lenght-percentage> or
  • the computed value of 'aspect-ratio' is not 'auto' and is not degenerate and the computed value of 'inline-size' or 'min-inline-size' is a <length-percentage>

Does that seem correct?

Edit: I now also found these properties: https://drafts.csswg.org/css-sizing-4/#intrinsic-size-override

[zcorpan]

Maybe a more general approach would be:

An element's concrete object size ignoring natural dimensions is its concrete object size but acting as if there are no natural dimensions, thus producing a rectangle with an absolute width and height.

This will match the actual concrete object size before the image has loaded.

Then, we can check that the "concrete object size ignoring natural dimensions" width is > 0, and any changes to this width is a relevant mutation.

[zcorpan]

I went with the approach described in the comment above. I believe it solves the problem discussed in #8008 (comment) and, at least in spec terms, doesn't introduce a lot of complexity.

[zcorpan]

(Need to rebase this after #8175 is merged.)

[zcorpan]

@yoavweiss PTAL :)

[zcorpan]

I'm a bit worried about compat for making more things "relevant mutations". Should we gate the new additions on usage of w descriptors in srcset?

[yoavweiss]

Should we also say that when the loading attribute is in the Eager state, the value of the sizes attribute must not be "auto"? Or are we cool with that meaning "100vw"?

[zcorpan]

It's disallowed here https://github.com/whatwg/html/pull/8008/files#diff-41cf6794ba4200b839c53531555f0f3998df4cbb01a4d5cb0b94e3ca5e23947dR29040

[yoavweiss]

What does that mean for a lazy-loaded image that was already loaded? If its dimensions change, it may get reloaded?

[yoavweiss]

Can you remind me of the motivation for this relevant mutations change? I found this, but it's not clear from that discussion what are the use cases this addresses. Apologies if we already went through this! :)

[zcorpan]

See my comment below.

[yoavweiss]

I'm a bit worried about compat for making more things "relevant mutations". Should we gate the new additions on usage of w descriptors in srcset?

I share that concern, and not sure that gating it would help significantly. I understand the motivations for loading attribute mutation changes, but unclear on mutations when the concrete element size changes.

[zcorpan]

The size observer is to keep the image "responsive" to changes in rendered size. For example, consider a gallery with thumbnails and clicking a thumbnail shows the same img as large as possible within the viewport. The environment hasn't changed, only the rendered width. It seems reasonable to assume that sizes=auto would not make the initial size "sticky".

[yoavweiss]

OK, I understand the reasoning to include the concrete size in the relevant mutations, and I think it makes sense to restrict those to only cases where they matter (so when srcset has 'w' descriptors).
Can you run an HTTPArchive analysis to see how often that condition is met?

IIUC, such content currently would be loaded once with a size of 100vw, and once we switch the the proposed behavior, multiple images may be loaded, if the image size is significantly increased or decreased. Is that understanding correct?