[meta] Web Compat Bucket

[jgraham]

Description

Many features that are already widely implemented and shipped have small implementation differences that cause compatibility problems on real sites. These aren't necessarily big enough to form a bucket on their own, but fixing them will significantly reduce site breakage going forward. This seems important to demonstrate an ongoing investment into the quality of the web platform.

If there is interest in this proposal we can propose specific bugs/tests as seperate issues. Proposed bugs to include can be validated with e.g. webcompat.com reports, or reports in bugtrackers relating to site breakage. To give an idea of the kind of thing I have in mind, here's an example proposal:

---

Description
Mouse events on disabled input elements work differently in Gecko compared to WebKit/Blink.

Specification
https://html.spec.whatwg.org/#enabling-and-disabling-form-controls:-the-disabled-attribute
whatwg/html#2368

Tests
https://wpt.fyi/results/html/semantics/forms/attributes-common-to-form-controls/disabled-elements-01.html?label=master&label=experimental&aligned but it seems to not cover this case.

Related Bugs
https://bugzilla.mozilla.org/show_bug.cgi?id=1653882
https://webcompat.com/issues/65773
webcompat/web-bugs#73699
webcompat/web-bugs#72988

[foolip]

@jgraham would you consider this a proposal in itself, or will this spawn other individual proposals? (Mostly wondering where on tomorrow's agenda to slot this.)

[jgraham]

Both (sorry!). I think the proposal is "have a bucket that's unifying theme is observed webcompat issues" but the actual selection of issues would require followup proposals.

[bkardell]

I think we should skip an overly global "bucket of bugs" idea unless we can kind of theme them somehow. Like, in your example, a goal could be to find things in a similar general area that involve something substantial enough to label and target... maybe just a bunch that are all generally related to lack of interop on interactive elements, for example, which play a huge role on the web?... Maybe this is part of #11 ?

[jgraham]

Well the theme would be "bugs that have caused user-noticable breakage on sites".

To expand out on that: although developers often complain about missing features, the interop problems we see affecting sites in the wild often aren't of the form "this whole feature is missing". Of course it can happen, but when it does it's often a specific decision by authors to not care about cross-browser support. In the cases where they care even a small amount, resources like caniuse.com are good at showing which features must be avoided entirely. This works even if when developers don't test in all implementations, or where there is extensive testing during the initial design process, but less once the overall architecture and technology choices are fixed and there are just incremental changes to the functionality.

This means that whilst missing features are the most obvious pain points in surveys, they are often not the kinds of bugs that end up as (for example) webcompat.com issues. Instead these tend to be cases where multple browsers implement a feature but with some smaller differences. These differences might go unnoticed by authors if they aren't performing rigourous cross-browser testing, or might be the kind of thing that developers work around during development, causing a general feeling that "cross browser differences make my life hard", but not being as easy to pin down as just "this feature isn't implemented".

Often we already have a web-platform-test covering the failure or bugs reported in various browser bug trackers. But because there are many failing tests, and many bugs reported, it's hard to know that these ones specifically keep causing compat problems. So from the point of view of improving the experience of web developers and end users, finding a way to identify and prioritise fixing these issues is at least as important as implementing new features.

[foolip]

For html/semantics/forms/attributes-common-to-form-controls/disabled-elements-01.html specifically, making sure it's part of #11 might also be an option.

[foolip]

https://github.com/web-platform-tests/interop-2022/labels/compat-bug-proposal is the list of issues.

[foolip]

@jgraham would you be able to compile a single list of tests that should be included here? I can then label those in wpt-metadata.

[jgraham]

Current status of tests:

IDNA

https://wpt.fyi/results/url/toascii.window.html

Text-Decoration

https://wpt.fyi/results/css/css-text-decor/text-decoration-subelements-001.html
https://wpt.fyi/results/css/css-text-decor/text-decoration-subelements-002.html
https://wpt.fyi/results/css/css-text-decor/text-decoration-subelements-003.html
https://wpt.fyi/results/css/css-text-decor/text-underline-offset-zero-position.html

event.path

https://wpt.fyi/results/dom/historical.html [Event.prototype should not have this property: path]

Mouse events on disabled input elements

[Missing]

onbeforeinput

Will be a subtest in https://wpt.fyi/results/html/dom/idlharness.https.html once whatwg/html#6743 merges

overscroll-behavior

Seems to be hard to test in detail?

Line-boxes don't contribute to scrollable overflow

[Missing]
Could port:
https://bug1637511.bmoattachments.org/attachment.cgi?id=9148082

Align-content on flex overflow

https://wpt.fyi/results/css/css-flexbox/abspos?label=master&label=experimental&aligned&q=flex-abspos-staticpos-align-content

Formatting context root with negative margin

https://bug-233956-attachments.webkit.org/attachment.cgi?id=446248
https://bug-233956-attachments.webkit.org/attachment.cgi?id=446249

@dholbert will port these

web-share self

[Missing]
Consider porting https://trac.webkit.org/browser/trunk/LayoutTests/http/tests/webshare/webshare-allow-attribute-canShare.https.html ?

[foolip]

@jgraham I've labeled the listed tests except https://wpt.fyi/results/dom/historical.html and https://wpt.fyi/results/html/dom/idlharness.https.html which require counting only some subtests.

[foolip]

@jgraham are there additional tests that need to be labeled (or written) here?

[foolip]

See interop-2022-webcompat for what is currently labeled.

I would prefer to drop the interop-2022-webcompat label from the flexbox tests, since those were already part of Interop 2021 and labeled as interop-2021-flexbox. Having tests belong to multiple buckets is technically not a problem, but it will probably occasionally lead to confusion when adding up the number of tests in the individual buckets doesn't equal the total number of tests.

[karlcow]

@foolip that sounds reasonable. Is there any plans to count the remaining interop-2021-flexbox on this round too?

[jgraham]

I created web-platform-tests/wpt#32351 for event.path being its own test. Still some more additions/refactorings needed to get full coverage of all the different parts of this.

[jgraham]

For the negative margin testing, @dholbert is planning to land the tests from https://bugzilla.mozilla.org/show_bug.cgi?id=1745310

[jgraham]

I created web-platform-tests/wpt#32367 for onbeforeinput.

[jgraham]

For mouse events on disabled form controls, I created web-platform-tests/wpt#32381 as a draft; we could land something there as .tentative. if we wanted to have a test, with the proviso it will be udpated once the spec is in place.

[foolip]

I've sent web-platform-tests/wpt-metadata#2406 to drop the flexbox tests, per #9 (comment).

[jgraham]

Remaining items here are:

• Land the tests from https://bugzilla.mozilla.org/show_bug.cgi?id=1745310 for formatting context root with negative margin
• Land the test in Add test for event propagation on disabled form controls wpt#32381 maybe as .tentative for disabled form controls
• Add a test for the web-share self permission. I don't know a lot about this; it would be useful if someone with more experience of the relevant spec could write the test.

[foolip]

@jgraham are all of the remaining items in #9 (comment) still remaining?

[jgraham]

Yes, I think so. I marked the PR as ready for review if you want to take a look. @dholbert and update on the negative margin tests?

[dholbert]

Yeah, sorry for the delay on those; I'll aim to get those tests landed this week, as a "part 1" to https://bugzilla.mozilla.org/show_bug.cgi?id=1745310. (I'll land them on mozilla-central, and then the wpt merge bot should upstream them soon after that.)

[dholbert]

Update: that negative-margin test is now in WPT:
https://wpt.fyi/results/css/CSS2/floats/floats-wrap-bfc-with-margin-001.html

Chrome, WebKit, and Firefox Nightly all pass it (with today's Firefox Nightly or later, which include the fix that I landed in https://bugzilla.mozilla.org/show_bug.cgi?id=1745310 ).

We can add that test to the interop2022 compat set. Since everyone passes it (now), it'll just be a minor "+1" bump to everyone's score, I guess. But it's still worth including since it covers a behavior difference that was a source of interop pain for Firefox until we got to the bottom of it and fixed this bug.

[dholbert]

I filed w3c/csswg-drafts#8142 to get this specced, FWIW.