Custom Elements: deferred upgrade until displayable #6480
Comments
zcorpan
added
the
topic: custom elements
@justinfagnani commented:
|
@zcorpan commented:
|
@zcorpan commented:
|
@mfreed7 commented:
@dvoytenko replied:
@mfreed7 replied:
@dvoytenko replied:
|
@zcorpan commented:
|
@dvoytenko commented:
@mfreed7 commented:
|
@zcorpan commented:
|
@justinfagnani commented:
|
|
Thanks for porting this over, and I like how it's evolved since I last saw it. In particular I love treating all types of visibility on equal footing, instead of special-casing
I disagree with this goal. Upgrade is always synchronous; it's just the process of establishing the right prototype chain, which the browser does automatically. There could be some separate lifecycle phase, such as "hydration" or "initialization" or similar, which is element-defined and async. But "upgrade" has a very specific meaning and it's synchronous.
This section doesn't really make sense to me. It seems like it needs more integration with the "Proposal" section. I can't really follow all the pasted-in comments so I'll trust you to summarize if there's any proposed changes that come out of them. |
|
I've added this to the agenda for the next triage meeting. @chrishtr fyi |
|
Some thoughts:
|
|
Yeah, this seems weird, since 99% of the time the upgrade will change rendering itself (either by creating or mutating dom). |
There could certainly be a version of this where
Take, for instance, the |
Hi Emilio, could you clarify what you find weird? Is it the whole idea of delayed upgrades, or something specific about the proposal in terms of custom elements? It seems quite natural to me to have a platform primitive that allows components to start their work only as they come near the viewport. Another way to think of it is that this is providing ways in the platform to easily achieve virtualization of content without a script-based polyfill (which is what many sites do today). Some sites even polyfill delayed custom element upgrades by using "stub" custom element class definitions that un-stub based on IntersectionObserver callback timing. Having this feature in the platform will:
To the point about upgrade-changing-rendering: as you say, the custom element upgrade is likely to change rendering state for itself during its upgrade. This will indeed mean that enough time will have to be reserved to avoid visual artifacts. But I view that as the cost of script-based content virtualization techniques. And we could imagine further enhancements to a delayed upgrade feature, such as platform APIs that say "delay, but do the work if there is idle time in which to do it", or specifying the desired distance from the viewport after which to start the upgrade process. |
I agree that all of these are concerns to be thought through carefully.
TL;DR I think we can follow the same method as we ended up with in There we defined the notion of skipped, which means "descendant of a The steps would be: Update-the-rendering for frame N:
Subsequent event loop task:
Update-the-rendering for frame N+1:
However, observe note 4 in the spec, which basically says that if the first time an element is considered for rendering (**) it's found to be not skipped, the above steps happen synchronously with the frame. In the case of the proposal being discussed here, it'd be this timing: Update-the-rendering for frame M:
Note that step 3 is at the same timing as For cases that are not
The second example above shows how to avoid a situation of FOUC / visual artifacts for elements rendered for the first time. In testing In other situations, such as scrolling, the CPU/FOUC concern is addressed by setting the margin of the (internal) IntersectionObserver to be a few viewports, which gives time for the UA to render. Now that there would be more script involved, that margin may need to be increased in some cases, or developers given more control over increasing it. (*) I think the spec may have gotten out of date in the numbering w.r.t the HTML, spec, will address that. |
I guess when I wrote this I imagined this as something that would run directly from the styling process, but your comment made me realize this is more subtle (more on the "IntersectionObserver" / "loading=lazy" camp, if I understand correctly). Is that the kind of model you had in mind? If so, that seems fine. I read your response above and it seems reasonable to do something like So for stuff that's just outside of the viewport, and assuming that stuff takes some reasonable size when not upgraded (which is something the page would need to take care of) then this works. But for stuff that's actually not rendered at all it is not quite clear to me how this would work without FOUC. Think something is If you do this based on Otherwise... at which point do you check whether this is "displayable enough"? Presumably after layout? Or just after styling? In any case, if the element is "displayable", then you need to run an arbitrary amount of content script, which might in turn invalidate all sort of style and make either more or less stuff displayable. So we need some sort of infinite loop prevention like Anyhow, my main concern at this point, then, is that with a naive implementation of what you describe above there seems to be potential for a lot of wasted styling / layout work. In particular, thinking of nested components, they'll upgrade once-at-a-time, which means that the cost of rendering is (at best) double. E.g., the parent component becomes "displayable", and its upgrade creates a shadow root with more components, which are not rendered yet, so they're not "displayable", so we layout and discover they actually are, and they get upgraded and create more components, and so on... That sort of pattern seems like it'd be pretty common (think, showing a |
Thanks for the quick response! Yes, it's the IntersectionObserver/loading=lazy mode. I agree the other one is unworkable. :)
I agree that
Nested Without delayed custom element upgrades, the sequence of actions would be:
Now it'll be:
It all happens in one frame due to the "first-render of content" rule in note 4 of the spec. Also note that I'm presuming we aren't so unlucky that b and c end up just past the IntersectionObserver threshold. This naively looks like three expensive renders with interleaved script, but in a good implementation the second and third renders are much cheaper, and the cost is not much higher than just re-rendering content-a and content-b. The reason we know it won't be very costly is that I'd say the total rendering time is likely about O(d^2 * n), where n is the number of DOM nodes and d is the nesting depth of custom elements. It's d^2 because each of the rendering passes are likely to have to walk down the tree to find the next thing to render, or walk up it to set dirty bits.
I agree that this cascading would be bad if it resulted in multiple rendered frames or big additional costs, but based on my analysis above it looks like that won't happen. |
But upgrading custom elements would more often than not change styles (due to the shadow tree rules affecting the host etc), right? So it's not as cheap as regular content-visibility, at best it also requires multiple rounds of style resolution, which is a bit concerning. |
Re host element: good point. Yes, the host element's styles could change, so that particular element may be have an extra style recalc after upgrade. But no other elements in the parent tree scope will change style, unless the custom element goes out of its way to do something terrible like injecting style sheets into the parent tree scope. The host element may also be laid out a second time (due to host styling in the shadow DOM), and this may cause changes to outer layout. Likewise, the removal of
Agree, it's not as cheap as One thing we could consider doing is to specify that any delayed upgrades trigger recursively. E.g. if you have the custom-c inside custom-b inside custom-a example I mentioned earlier, if custom-a upgrades, we also upgrade custom-b and custom-c at the same time, even if we're not they will be come unskipped, and don't know if they are |
|
It is strange to me that deferring upgrade is tied to the entire class of custom elements. In lazy imaging loading, for example, we added a content attribute on It is also strange that the proposal seems to make It also seems like the proposed semantics can be easily polyfilled if scoped custom element registries were added since whatever custom elements that want to defer upgrading can simply create a new registry and not define any custom elements in it. |
|
There are browser features that assume DOM represents all of page's content. A) hash links: browser will search DOM to find an Element matching hash-link. Frameworks have already implemented variants of lazy loading (ex: infinite scroll). The lack of interoperability with above features has been an issue (ex Notify page that "find in page" is being used, w3c Find-in-page API proposal ). Indiscriminately using deferred upgrade for performace benefit would be a hazard for developers. Most do not test the above features specifically, they just work. Deferred upgrade would break them. I do not have a strong intuition on how to solve this. The obvious solutions are:
|
|
@atotic I think the proposal here should fit very well and be fairly error-proof. To start with In a way this proposal fixes the issues like infinite scroll and other lazy loading concepts that you mention. |
|
@dvoytenko you are correct, the only scenario where "text is not in DOM" would present problems is I've looked over the hidden-matchable proposal. It assumes that DOM contains all the relevant text. Custom elements deferred upgrade combined with "content-visibility:auto" is the only combination of features that results in "Incomplete DOM until upgrade". Do any of the current proposals address this scenario? |
In many cases, the searchable content is in light DOM that is slotted into the custom element. Therefore it's there in the DOM. See below for an example. Right now, there is no such thing as a "registered but not upgraded" custom element (because there is no deferred upgrade mechanism). There is only "not registered" and "upgraded". Here is what happens in those situations: Not registered:
Therefore the content will be searchable by find-in-page etc. Upgraded: Content is searchable, because the slotted content is distributed. The new state is: "registered". What should happen? I think there are two cases: a. Shadow DOM is attached (via declarative shadow DOM) Case b is potentially bad, because undistributed content is like display:none in terms of searchability. However, if there is no shadow DOM then the element will behave like a so we're all good. Right?
|
If the content is in light DOM, there is no problem. The issue only exists if text nodes are created by custom element. For example: If These are not common use cases, but web developers have a habbit of doing things we did not anticipate, and expecting it to work. It'd be nice if we had an answer. It might be as simple as "All your text should be inside light DOM" |
I think the answer is:
|
Note, too, that the proposed design is configurable per Custom Element subclass (pending discussion about how static this needs to be, see #6480 (comment)). If a For CEs that don't need to be upgraded to be searchable (either through light-dom SSR or declarative shadow), they can use |
|
The main problem with lazy loading and init is that there may or may not be additional networks requests which are async again so out of my view there is already lazy loading when you for example write a async connectedCallback that applys lazy stuff on the element (this) |
|
There are a few comments here about how, in a completely generic case, the deferred upgrade could lead to bad results. However, to stress, this feature is designed to be completely opt-in, either by a component publisher, or a component user, or both. We can tweak the proposal to make it more restricting in this case. The goal is to enable the components that are ideally prerendered and free of other side-effects from taking up too many resources before they are shown. Notice that the way this proposal is currently defined, to be deferred these components must be either in |
|
@frank-dspeed wrote:
Do you mean that your custom element loads additional network data after being connected? An example would help. @dvoyenko wrote:
Deferred upgrade is a tool that can be used to optimize page loading. The optimal, intended use, would be for a page to defer loading of costly non-displayable custom elements. Optimal use would also avoid nested deferred loads, because nested loads cause multiple layout passes, and make loading performance worse, not better. Like many performance optimization tools, it needs to be used carefully. To decide whether custom element should be loaded lazily, one should know: Which leads to an interesting conclusion: B) implies that the page author is best equipped to make the decision whether a custom element is lazily loaded or not. |
|
B) is easily handled by the class XFoo {
static deferredUpgrade(element) {
return !element.hasAttribute('loading') || element.getAttribute('loading') === 'lazy';
}
}The proposed API is flexible enough to overcome all the bad cases mentioned in this thread, which is part of what Dima is trying to stress. |
|
@jridgewell wrote:
I am not seeing how |
|
You missed the second half of your B) "This, in general, is only known by the page author.". |
|
@atotic Here is a example custom-element that defferes everything that it does exempt executing the upgrade as soon as the element is created /**
* Returns a Promise that resolves as soon as the
* Element comes first time into viewPort
* @param {HTMLElement} el
* @returns {Promise<void>}
*/
const onDisplayAble = (el) => {
return new Promise( (resolve) =>{
const root = null;
const threshold = 0.1 // set offset 0.1 means trigger if atleast 10% of element in viewport
const observerConfig = { root, threshold };
const observer = new window.IntersectionObserver(([entry]) => {
const isDisplayAble = entry.isIntersecting;
if (isDisplayAble) {
resolve();
}
}, observerConfig);
observer.observe(el);
});
}
const startAsyncUpgrade = async (el) => {
// do something with the element after setTimeout
// do something with the element after listiningForEvents
// do something with the element after networkRequests
onDisplayAble(el).then(()=> { /* do something with el */ })
}
customElements.define('my-app', class extends HTMLElement {
connectedCallback() {
startAsyncUpgrade(this);
}
})Bonus Polyfill for the observer part /**
* Returns a Promise that resolves as soon as the
* Element comes first time into viewPort
* @param {HTMLElement} el
* @returns {Promise<void>}
*/
const onDisplayAblePolyfill = (el) => {
return new Promise((resolve) => {
const handler = (el) => {
const rect = el.getBoundingClientRect();
const isDisplayAble = rect.top >= 0 &&
rect.left >= 0 &&
rect.bottom <= (window.innerHeight || document.documentElement.clientHeight) &&
rect.right <= (window.innerWidth || document.documentElement.clientWidth)
if (isDisplayAble) {
resolve()
}
};
if (window.addEventListener) {
addEventListener('DOMContentLoaded', handler, false);
addEventListener('load', handler, false);
addEventListener('scroll', handler, false);
addEventListener('resize', handler, false);
} else if (window.attachEvent) {
attachEvent('onDOMContentLoaded', handler); // Internet Explorer 9+ :(
attachEvent('onload', handler);
attachEvent('onscroll', handler);
attachEvent('onresize', handler);
}
})
}ConclusioncustomElements are simple pre defined selectors thats it. So they are similar to a on insert event and so should be Sync. |
Oh, I missed the place where you agree that page author being in best position to decide. We are fully in agreement. If page author being decision whether a custom element is lazily loaded or not, the next question is: The proposal is for custom element class to include static method: How would page author do this, if they do not have control over the element implementation? What do you think about:
my-element {
custom-element-upgrade: eager | lazy;
} |
Isn't this the same as the author using a custom subclass of a library's custom element? class MyElement extends LibElement {
// Override `LibElement.deferredUpgrade` for my use case
static deferredUpgrade() { return false }
}
customElements.define('my-element', MyElement);
A new CSS rule to opt-into deferred behavior would be fine with me. |
|
Making element upgrade depend on a CSS property doesn't seem like a good idea to me. That would cause style updates before every custom element upgrade. |
How?
|
What guarantees that? But anyhow the case I'm concerned about is not the property dynamically changing. When insert an element into the DOM, you don't style that element until you need to (either because of CSSOM APIs, or because you want to render the page to the screen). However in order to know whether the element needs to be upgraded synchronously, you need to style the element right on insertion, individually (because it might be eager). And because of how inheritance works you also need to update style of all its flattened tree ancestors. So it'd be a massive performance blunder, unless I'm missing something. |
|
Thanks for the long explanation, @emilio. That settles the issue, CSS is not suitable for marking elements for deferred upgrade, because upgrades might happen before styles are available. I've just moved from Layout to DOM team, and appreciate your patience with my newbie questions. |
|
To reduce dependencies of upgrade decision making, IMHO the two options I see working best are: a. A The (a) might end up better since it will give more control to the page author. It's also similar to |
|
I discussed some more with @atotic today. A summary of our current thinking is below.
Therefore I propose an HTML attribute:
|
|
Regarding the question of why add this attribute rather than a polyfill from developers: this is a method of a developer quickly and effectively improving performance of their existing pages, without having to optimize it from the inside-out. It's similar to the reasons why In addition, I expect a polyfill will be less performant, because it still requires executing javascript that can otherwise be more easily deferred. |
|
Another one to consider is an |
|
i still see this in the domain of the component author who needs to choose if he can implement his algos async or sync then he can offer to the user a Observed Attribut or even none observed and check on connectedCallback if the attribut is set to lazy or something else and this way give the page author the power to choose to lazy load or what ever loading algo is supported by the specific component implementation i for example code really complex WebRTC Components with a lot of interactions with multimedia elements and also workers and audio worklets a lot of processing and all that also remote for example codec selectors and all that. I really know out of production what edge cases are there for example registering audioWorklet processors async then establish signalingChannels async and so on. a single component that does only one thing can have easy 1600 loc (lines of code) and i do long one liners. When you for example create a simple Component nothing complex like i do with a lot of async dependencies there is never a need to do async on a single component level. the css auto property to lazy render already deffers the execution of the connectedCallback
Maybe edge case to consideri create often customElements as container they assign connectionPromises as propertys to the component so that all nested components get the propertys from the used parent component. when a developer (page author) now lazy loads that everything inside it will break when that upgrade would happen async all components would not be able to grab the propertys. even the ones that are not loaded. also everything that uses it will break we now need to handle enforced sync upgrade as edge case. |
|
I like the direction this is going @chrishtr I'd like to reiterate my desire to see this considered within a broader context of lazy loading/upgrading use cases, especially since lazy loading is rapidly becoming a primary feature of frameworks like Wiz (Google internal), Astro, Qwik, and work my team is doing. The two general axes these frameworks sit on are:
I know event buffering is too large of a topic to cover here, but I wonder if we could consider how these attributes would interact with WICG/webcomponents#782. Specifically, if a developer wanted to load and upgrade a component lazily, would IOW, what happens if we have: <script type="module">
customElements.lazyDefine('x-foo', async () => (await import('./x-foo.js')).XFoo);
</script>
<x-foo upgrade="lazy"></x-foo>Does that make sense? Would you need both, or could the definition have an option for loading on content visibility rather than element creation, eliminating the need for the attribute? |
|
@justinfagnani i see lazy define as near impossible at last the element constructors need to get finished else it will break this class myEl extends HTMLElement {}
new myEl() // illegal constructor
customElements.define('my-el', myEl)
new myEl() // works will produce my-el tag(as your also working on scoped registries only for your interrest scoped registries also breaks that) new myEl can only work when the HTMLElement and constructor gets completed via the customElement Register Call ok after more thinking sure it is possible but then we introduce extra code that is needed to watch when what defines what i see always abstraction over abstraction that makes me code more and more to handle the new introduced cases. |
This was first written by @dvoytenko in https://docs.google.com/document/d/1KNVZXlw-wygoNwZz0m_105ISBSjN1Q-sKh1bkw4CgPw/edit# , moving it here as suggested by @chrishtr to facilitate broader discussion. cc @justinfagnani @mfreed7
I'll repeat the existing open comments in the doc below.
Background
Custom Elements API is a mature Web spec that allows DOM elements to be upgraded with custom styling, markup and behavior. It’s a convenient and performant solution for many use cases. However, this spec lacks any notion of “lazy instantiation”. The browser upgrades a defined element as soon as it’s parsed. This could be a bottleneck for applications that rely heavily on custom elements. This document explores an idea of allowing a natural lazy-upgrade mechanism for custom elements.
Goal
Extend the Custom Elements spec to enable lazy upgrade and instantiation of defined custom elements. The expectation is that it will reduce initial CPU and memory usage of a page that relies on custom elements.
Proposal
The proposed solution is to defer upgrade of custom elements that are not displayed on-screen or likely to be on-screen soon, and/or have CSS which make them invisible without a change to DOM state. Let’s call these conditions displayable.
Examples of non-displayable elements:
display:nonestyle orhiddenattribute.visibility:collapsewhen space is removed (tables and flex children).content-visibility:auto|hiddensubtree.By default a custom element upgrade will be unchanged. However, the new APIs will allow an element to opt into the deferred upgrade mode. Such an element will not be immediately upgraded by the browser. Instead they’d be queued up and upgraded when it becomes displayable.
Proposal details
/1/ The custom element class can include static methods to indicate that it desires deferred-upgrade behavior:
When absent, the
deferredUpgradeis assumed to returnfalse, i.e. the current upgrade behavior.See the “Deferred upgrade of self” section for more information on
deferredUpgradeSelf./2/ The parser will recognize a “deferred-upgrade” element, but will not immediately upgrade it. Instead it will schedule the upgrade for later, when the style tree is known and thus
display,content-visibility, and other relevant values have been computed in respective DOM subtrees. These are “pending-upgrade” elements. The queue of “pending-upgrade” elements is checked when the relevant styles change and, if becomes displayable, the respective elements are upgraded. Ideally they would be upgraded respecting the CPU resources to avoid long tasks./4/ The new
CustomElementRegistry.whenUpgraded()API will be added, sincewhenDefinedcan no longer be used as an indicator that an element has been upgraded:/5/ The existing
CustomElementRegistry.upgrade()API can be changed slightly to force upgrade for “pending-upgrade” elements. Its semantics are already mostly compatible. However, one change we could apply: it should return a promise for when the upgrade is complete./6/ A stretch goal: provide
:upgradedpseudo-class/state in addition to the existing:defined. This state will be set when the element has upgraded. Seeupgrade()andwhenUpgraded()above./7/ A stretch goal: provide an additional callback to the Custom Element class, in addition to the connectedCallback and others - the upgradeCallback. It’s not uncommon for an actual upgrade to be asynchronous. This callback will return a promise that will return when the upgrade has completed. See
whenUpgrade()andupgrade()methods./8/ A stretch goal: provide an additional attribute-backed property to disable deferred build behavior.
It’s a stretch goal because with the proposed
deferredUpgrade(Element)API, the callback can check this attribute and return the appropriate response. However, it’d be beneficial to be consistent. And additionally, the browser can handle mutations for this attribute/property automatically.Deferred upgrade of “self”
This section is mainly relevant to how this proposal works with the
content-visibility: hidden. Thecontent-visibilityapplies to an element’s DOM subtree - not the element itself. However, for many custom elements, if the contents is not rendered, it doesn’t make sense to upgrade the element itself. For such elements, it’d be also beneficial for overall performance to defer their upgrade. This is why the proposal includes thedeferredUpgradeSelf()method.The text was updated successfully, but these errors were encountered: