Tests for directionality and shadow DOM

[meyerweb]

40 tests and equivalent rendering references, with the tests adapted from various links posted in whatwg-html issue #3699.

[meyerweb]

@bkardell pointed out that Safari is failing these tests even though it really shouldn’t, due to the use of :dir(). See, for example, this test result which (if you disable the diff highlighting) shows Safari would have passed except for the generated content that only gets changed if :dir() is supported.

One outcome of the most recent WHAT WG triage meeting was that these tests need to be moved to a different area of WPT anyway, so I’ll bundle the test rework with the move to the different area. Stay tuned.

[meyerweb]

The tests should now only be reliant on implementations’ Shadow DOM and directionality behavior, plus a scrap of very basic JS. They’ve also been relocated as requested by WHAT WG. Absent any errors of assertion in the tests themselves (e.g., expecting LTR where it should be RTL), I believe these tests are ready.

[bkardell]

Just summing up where things stand to hopefully make it easier to focus on where we need discussion/desictions. We have here 40 tests which set up various scenarios for shadow DOM and test the dir reported by getComputedStyle on some element. Of these, as of today..

• 31/40 tests where we have tests/interop agreement from the 3 engines (1,2,3,4,5,7,8, 9,10,11,12,13,14,15,16,17,19,20,21,22,23,25,26,27,28,29,31,32,35,37,40)
• 7 more where we have agreement in 2 engines and the test, but Safari disagrees (18, 24, 30, 33, 34, 38, 39).
  • I think the first question is can we verify that these are just bugs and move them into the "done" colum, or is there some actually disagreement? @rniwa ? If we can agree these lone holdouts are bugs, we're all the way up to agreeing on the results of 37/40 tests and a long way to interop on them....

Finally, what this leaves for more difficult challenges is only 2 items... On both 6 and 36, both FF and Safari currently disagree with Chrome and the test. Which is right, and why? These harder ones are:

1. dir-shadow-06.html

This is using dir=auto on the slot element and a paragraph inside containing some hebrew text. The test checked the computed style of the paragraph projected into the slot. Chrome and the test suggest this should be rtl, FF and Safari report LTR.

2. dir-shadow-36.html  

A host with one child slot. The slot itself has dir=auto and some default hebrew text, but projected into it is a text node with an english word. The test operates by calling getComputedStyle on the slot. Chrome and the test believe the correct answer is LTR, but FF and Safari say it is RTL.

[meyerweb]

In addition to @bkardell’s fantastic summary of the test results, I want to add what I understand from the test assertions themselves, so we can find out if they’re the behaviors people actually want. In summary:

Explicit dir values on shadow hosts, slots, or elements that are slotted will hold sway over the content within, whether it is light-tree or shadow text.

Thus, if dir="ltr" is set on a shadow host, and the slotted content is a paragraph containing Arabic, the directionality of the Arabic will be LTR instead of its inherent RTL. And analogously for dir="RTL" filtering down to English to make it RTL instead of its inherent LTR. This is true whether or not the text content has an enclosing element between it and the shadow host.

Explicit dir="auto" values on shadow hosts, slots, or elements that are slotted means the content within will act according to its inherent directionality, whether it is light-tree or shadow text.

This demonstrates there is no inherent directionality in slots, shadow elements, or light-tree elements. Absent a directional dir value, content asserts its inherent directionality, and dir="auto" does not cause a specific direction to be imposed on the content.

[meyerweb]

An addendum/clarification from @MyidShin:

Regarding dir-shadow-036.html, AFAIK, we shouldn't violate encapsulation of shadow DOM and should resolve the light tree element's directionality in the light tree and do not walk into the shadow tree for encapsulation (as per whatwg/html#3699). So, dir-shadow-036.html's result should be 'LTR' and Chromium fixed recently this issue at https://chromium-review.googlesource.com/c/chromium/src/+/3066807.

The dir-shadow-036.html does currently look for LTR, so as long as the above behavior is what’s desired, the test is good to go. However, the above does address something my previous summary may have elided.

[rniwa]

What's up with 8 lang attribute tests without reference files? Maybe inadvertently included in the PR?

[rniwa]

• 7 more where we have agreement in 2 engines and the test, but Safari disagrees (18, 24, 30, 33, 34, 38, 39).

  • I think the first question is can we verify that these are just bugs and move them into the "done" colum, or is there some actually disagreement? @rniwa ? If we can agree these lone holdouts are bugs, we're all the way up to agreeing on the results of 37/40 tests and a long way to interop on them....

No, these are not WebKit bugs. On the contrary, according to @fantasai's definition in whatwg/html#3699 (comment) to which I had previously agreed, WebKit's behavior is correct and the tests and Firefox/Chrome are wrong. Specifically, dir=auto should resolve in each tree without crossing shadow boundaries first. Only then should direction property inherit from a parent to a child in the flat tree.