Define how the core browser set was determined

[tomayac]

Based on an interesting question asked on Mastodon by @dgp1130. I was intrigued, but couldn't find based on what criteria the core browser set was determined.

[ddbeck]

Hi @tomayac, I'll do my best to answer your as succinctly as I can. I'll respond to the more specific "What if…?" question separately.

In the strictest terms, the core browser set was selected by the web-features owners group in a consensus-seeking decision-making process. We worked on the overall definition, including the browser set and releases timeline, until we reached an actual consensus. This process took about 6 months. Most discussions happened in WebDX meetings, the WebDX matrix chat, and in issues labeled here.

More broadly, we picked the browser set to accommodate a number of requirements and constraints:

• Developer expectations (e.g., through surveys, direct feedback on an earlier definition, and representation in the owners group)
• Developer behavior (e.g., looking at, for instance, open source browserslist configs)
• Baseline consumer expectations (i.e., what would work for caniuse and MDN)
• Upstream data availability and quality (i.e., what data sources were actually available to us, for support and usage)

A lot of work went into trying to find some statistical rule (e.g., using StatCounter usage thresholds) that would encompass everything. Unfortunately, data quality and other issues prevented that from happening. So the governance group reviewed all the information and made a judgment about a browser set that we hoped would be satisfactory to developers.

The formal Baseline definition document contains a lot of information about our shared motivations, at the time it was adopted.

[ddbeck]

To @dgp1130's question, which I'll excerpt here:

Presumably the browser set would evolve if a new browser gets enough market share to justify its inclusion.

What happens if that browser is missing a particular feature supported by the other browsers?

Does that imply that Baseline 2025 could hypothetically include less features than Baseline 2024 if it included a new browser? I feel like losing features is something you wouldn't expect in Baseline, much like you don't expect a version to go backwards.

Is there any expected outcome for this particular scenario?

When we adopted the Baseline definition, we did not resolve to answer the question of what the meaning of Baseline would be before 29 July 2015 (the first release of Edge) or after some future change to the core browser set. We talked about it and speculated on it but, if I remember correctly, the consensus was that it would be premature to work out a definition versioning scheme or transition policy before we had demonstrated that Baseline was useful to developers.

We did commit to an annual review of the definition, to affirmatively decide whether it was still working or to kick off a revision process. Since being unsurprising is part of Baseline's goals, I can't imagine we'd make a change to the core browser set without a lot of advance consultation and notice.

I want to note one last thing, though:

I feel like losing features is something you wouldn't expect in Baseline, much like you don't expect a version to go backwards.

It is possible for a feature to move backwards, even with the existing browser set, though it should be a rare event. For example, suppose a feature reaches Baseline widely available status but a browser subsequently unships feature (or ships a bug so serious that it's close to the same thing). We'd have to "un-Baseline" that feature and the feature would have to progress from limited availability through newly available again. Thankfully all of the examples I have for this, such as IndexedDB, are increasingly historic—but there's no rule that says it can't happen again.

[dgp1130]

Thank you for this context @ddbeck and thank you for following up on my question @tomayac!

What you describe around losing features does sound reasonable. SharedArrayBuffer also comes to mind as an instance where a feature was unshipped cross browser, though like you mention that's a pretty rare event on the web which isn't too much of a problem in practice. It does make sense that if the browser platform moves backwards by unshipping or breaking a feature, then Baseline does need to align with that, even if it might be counter-intuitive.

So if I'm understanding correctly, it sounds like exactly how a new browser would be introduced into Baseline is still somewhat ambiguous and undefined and will likely stay that way until such a scenario actually happens? That sounds reasonable given the context and the way these kinds of decisions tend to get made, though I just want to point out that this does have an ecosystem impact.

Any library or tool which commits to Baseline support ("We will only require browser APIs which are Baseline as of any given version's release date") doesn't really know what it's committing to in the situation of a new browser addition. As you mentioned, Baseline can already "go backwards", but those cases are relatively exceptional. New browsers are somewhat common, but reaching the threshold for inclusion in Baseline is likely just as, if not more exceptional than unshipping / breaking existing features. So I don't think this is a major problem, but it does mean that any ecosystem tool adopting Baseline as a browser support policy needs to consider what happens in the scenario that a new browser is added. With that currently being undefined in Baseline itself, it's hard to know exactly how to handle that case. The most likely option for a versionable tool would be something like "We will only require browser APIs which are Baseline as of any given version's release date, with the exception of versions X1-X2 which do not support browser Y."

All that said, a new browser would certainly be an exceptional circumstance and any versioned tool can adjust its browser policy to match since Baseline support is tied to a specific date, so adding a new browser would presumably not retroactively change Baseline for previous dates. As a result, I don't think this is a major problem, but something to be aware of. Maybe it would make sense to document somewhere that this scenario is still undefined? Or maybe at least commit to not changing the definition of Baseline support for dates prior to inclusion of the new browser (that part feels uncontroversial to me and would help at least versioned tools to adopt Baseline). But otherwise I can imagine that we likely won't see consensus on actually defining it until this scenario really happens.

Thanks for the discussion here, this is a really interesting edge case and I appreciate the added context!

[ddbeck]

Briefly, summarizing my to do list here:

• Document undefined-ness of Baseline in the eras before and after the present core browser set (Baseline defintion: flag undefined status situations #2142)
• Document guarantees (if any) in the case of adding a new browser to the core browser set

---

Maybe it would make sense to document somewhere that this scenario is still undefined?

Absolutely. I've opened #2142 to (begin to) cover this.

Or maybe at least commit to not changing the definition of Baseline support for dates prior to inclusion of the new browser (that part feels uncontroversial to me and would help at least versioned tools to adopt Baseline).

Yes, I think it's a good idea to make some sort of guarantee about this, even if it's minimal (e.g., some sort of notice period before making changes to the core browser set). As for the specific shape of that guarantee:

Do I understand correctly that you're suggesting that we commit to freezing the browser set for a feature at the time it reaches Baseline status? That is, once a feature reaches Baseline, we'd only score that feature on the (ongoing) support from the core browser set at the time?

If I have that right, I'm not sure if we can make that strong a commitment. Over the long haul, I'd expect that if developers cared enough about the new browser to warrant its inclusion that they'd also eventually stop relying on features that don't work in that browser, no matter the historic case for marking that feature as Baseline. But that's an assumption and one I'd like to test a bit. I think there are other plausible routes (e.g., a sunset period or an explicit "legacy" or "vested" status for features that have this quality) and I'd like to do more intentional research into developer expectations about all the options before committing to one.

An easier commitment today might be be something like: introducing a new browser won't cancel a Baseline status for an existing feature for at least 30 months (e.g., an off ramp that mirrors the widely available time horizon). Telegraphing the removal of a feature from Baseline—if we know it's going to happen in advance—seems really valuable.

This is somewhat related to #311 and #955, where we need the ability to communicate a probable future status (i.e., shipping or unshipping anticipated in a future release). We'll actually have to resolve that sooner than any change to the core browser set, since feature deprecation or removal is a rare but more frequent than the appearance of a new must-support browser.

This is also related to #1971, where we should explain (as part of the data) why a given feature has regressed, if it does.

[tomayac]

(Just wanted to thank @ddbeck and @dgp1130 for the discussion. I'm an interested silent observer [and the messenger in this case], so don't block on a response from me.)

[dgp1130]

@ddbeck, I think you might be interpreting this support question in the context of supported features, but I'm actually thinking about this from the set of supported browsers. I'm looking through the lens of "What are the Baseline browser versions?". This is a slightly different question from "What are the Baseline features?", which is a separate artifact that is derived from the set of browser versions. For libraries and frameworks, the goal is partially to identify what features are available in Baselines, but also to identify what workarounds might still be necessary to support Baseline browsers. So in this context, I think I care less about feature support and more about the precise set of versioned browsers at a particular time.

Let's look at this from the context of a strongly versioned library/framework released on a particular date. My thinking is that such a tool might use the "widely available" Baseline by defining its browser support as:

Version ${V} of this library/framework/tool supports the "widely available" Baseline as of date ${D}.

Date ${D} is likely the release date of that version of the tool, or close to it. In practice, this means that the "widely available" Baseline would take ${D - 30 months} and get the then-latest versions of the core browser set (call this ${B}). The minimum set of features supported by those browsers provides the Baseline feature set (call this ${F}).

The above support policy essentially states:

We support all browser versions ${B} or above and will only require browser features in ${F}.

This feels reasonable to me on the surface. However it gets tricky if a new browser is added and regresses the feature set. In theory, any previously released versions of a library/framework should be isolated from that change, given that they only committed to supporting that "widely available" Baseline at their release date (browsers ${B} and features ${F}).

So my ask for the Baseline definition would be to version it in a similar fashion. Say a new browser is added to the core set on date ${D}. The exact semantics on this are undefined and TBD as you mentioned, but I'm wondering if Baseline could at least commit to not changing the definition of ${B} and ${F} for date ${D - 1} (and earlier). If a new browser is introduced, it should only affect Baseline at some particular start date and the dates after, but no dates before. This allows versioned tools to rely on a stable definition of Baseline at a particular date.

This would still give you the freedom to dictate exactly when and how a new browser is added while allowing the ecosystem to use Baseline without fear of a new browser addition undermining their existing released versions. I expect this might be a more academic concern as in practice I don't expect a large proportion of users would file bugs against their libraries/frameworks/tools saying "Please add support for this newly included browser to all your existing released versions because Baseline has been updated and that retroactively applies to all your prior releases." Most maintainers are just going to close those kinds of issues, but it would be nice if the precise wording of such a browser policy did match the way most humans would be likely to interpret it.

[captainbrosset]

This is related to a discussion we had on Sept. 26, 2024, at the WebDX meeting, about including a WebView into the core browser set. I'm pasting the minutes below as this is relevant here.

What would it take to extend Baseline to cover WebViews?

Francois: A few days ago in the WebDX CG there was a discussion to extend the Baseline concept to cover Web Views. Ben and Niklas are here to present the idea.

Ben: Webviews cover a large part of web traffic, devs might not think about that as a surface that they might end up on, but that’s the reality. We are working in the WebViews CG to collect data on Webviews. Both Android and WKwebview are now in BCD. We’d like to understand what the expectation would be to add WebViews to Baseline. We’re okay to do the legwork.

Francois: Some of the discussion was about the transition from today to that new world. Things that were Baseline before would stop being Baseline.

Kadir: Transitioning is a very good point. We talked about an initial browser set, with the need to check regularly whether the set needs to be revised. The baseline dates are not carved in stone. Saying that is one thing. Making a change is a completely different thing. One requirement: to align on a set, we ran a survey. We did that because we wanted to end up with a definition that would match developers expectations. We knew that WebViews were a significant part of the web, but they did not appear in the survey. For the baseline definition to change and include WebViews, developers need to have a mental model that they won’t use a feature until it becomes available in WebViews.

Dom: From a WebView CG perspective, we need to significantly increase the awareness of developers that they have a problem that they don’t know they have.

Ben: If I have Webiews in BCD, how easy is it for me to compute things?

Daniel: If you wanted to add a browser, it’s 4 lines of code, roughly. The hard things are getting data in BCD in the first place, and tracking what features it would affect.

Ben: We also have “Can I WebView”. Florian pointed out that seeing the delta would be useful. If there’s some automated way to compute things, we could present the delta.

Niklas: Most developers aren’t aware that their applications run on WebViews, e.g., because users click on a link through Facebook. They may be interested in Baseline but unaware.

Patrick: WebViews and PWA, the story is roughly the same. Very large part of the traffic. We never really hear from these folks in the surveys. I think that’s because these apps are very few in numbers but very large in traffic. I don’t see any other way around if we change the set of browsers to setting baseline in time: Baseline 2024 is based on this set of browsers. Baseline 2025 is based on this other set.

Kadir: Good point here, already the case with 2012, first version of Edge. One of the big issues that we looked into was how long it takes to get to 95% of users. One difficulty with WebViews is that it’s harder to have actual data on that. Can you tell that you’re actually running in a WebView? That would be something to clarify as well. Especially for the widely available.

Ben: That’s a good question.

Kadir: It would also be interesting to see the delta not only in terms of BCD keys but also in terms of web features. It might also be a way to generate that awareness that we were talking about.

Thomas: If you have a WebView and you know that this app is using this WebView, the app may decide to disable certain features. I don’t think that can be measured.

Dom: That’s a big part of bringing awareness. Being able to detect that it’s something they should be dealing with.

I think that versioning Baseline is something that would need to happen if we, one day, add a new browser/webview/implementation to the core browser set. Which I think is what @dgp1130 is saying here:

So my ask for the Baseline definition would be to version it in a similar fashion. Say a new browser is added to the core set on date ${D}. The exact semantics on this are undefined and TBD as you mentioned, but I'm wondering if Baseline could at least commit to not changing the definition of ${B} and ${F} for date ${D - 1} (and earlier). If a new browser is introduced, it should only affect Baseline at some particular start date and the dates after, but no dates before. This allows versioned tools to rely on a stable definition of Baseline at a particular date.