Add harness input/output

[mzgoddard]

Preview Tests

• Add TestRunInputOutput class to gather various test information in a test html file.
• Move classes out of aria-at-harness so they can be used by dependents of aria-at.

TestRunInputOutput Summary

The main change in this PR adds the TestRunInputOutput class. It takes Input helpers instances to produce the state for a TestRun instance and options for a TestWindow instance. Each Input type has a public interface representing some current information processed currently at runtime.

This PR includes static methods on those inputs that process currently available information in the test html files for that public interface. This design is intended to provide a means to iterate the input data format for a test. The current data has a diverse set of format and is collected in many distinct methods, making it difficult to use and understand.

After this PR we will add new implementations of the Input types and process improved data formats while maintaining backwards compatibility with the existing formats.

New Files

• aria-at-test-io-format.mjs
  • class KeysInput
  • class SupportInput
  • class CommandsInput
  • class ConfigInput
  • class ScriptsInput
  • class UnexpectedInput
  • class TitleInput
  • class BehaviorInput
  • class PageUriInput
  • class TestRunInputOutput
  • class TextRunExport
• aria-at-test-window.mjs
  • class TestWindow

Inputs

TestRunInputOutput currently consumes 9 Input types which can be broken into two groups: standalone and dependent. Standalone inputs can be processed on their own. Dependent inputs require one or more other inputs to be able to process their information.

• TitleInput, a standalone input. The title is the title of the test and originally appears in tests.csv. In this PR it is sourced from a test html file's title element.
• PageUriInput, a standalone input. The page uri is the reference page loaded in the test window and originally appears in references.csv. In this PR it is provided as an argument to one of the aria-at-harness functions.
• UnexpectedInput, a standalone input. The set of unexpected behaviors listed in the instruction document. In this PR it is builtin aria-at-test-io-format.
• ScriptsInput, a standalone input. The set of executable scripts currently used to setup the reference page after it is loaded in a test window. In this PR it is fetched with the global scope member scripts if it exists.
• SupportInput, a standalone input. The support information provided by tests/support.json.
• ConfigInput , a dependent input. The configuration sets things like which at a test is being performed with for tests that are defined for multiple ATs. In this PR the config comes from the page's url query parameters and depends on SupportInput.
• KeysInput, a dependent input. The keys are a map of key ids to keystrokes a user will perform and some other key related data. In this PR this keys comes from the keys module and depends on ConfigInput. Importantly keys does not depend on SupportInput. The keys implementation in this PR could be provided a ConfigInput that does not need SupportInput to be instantiated.
• CommandsInput, a dependent input. CommandsInput represents the information currently found in commands.json. In this PR it depends on ConfigInput and KeysInput to process the information provided by commands.json.
• BehaviorInput, a dependent input. BehaviorInput represents the information currently found in a test.json file. In this PR it depends on CommandsInput, ConfigInput, KeysInput, TitleInput, and UnexpectedInput to process the information in a test.json object.

Processed Outputs

When TestRunInputOutput has the right inputs provided it has some methods that can be called to provide data in a format need to render test instructions, record test results, open a test window, and create the test result submission object.

• testWindowOptions() depends on BehaviorInput, PageUriInput, and ScriptsInput. It returns the options for a TestWindow instance.
• testRunState() depends on BehaviorInput and ConfigInput. It returns an initial state for TestRun instances.
• submitResultsJSON(state) depends on BehaviorInput. It takes a current state from TestRun instances, and returns the json object submitted for a aria-at test.

[howard-e]

Having worked a bit closer with the implementation this has been built up on, I can confirm what here works quite well and I can certainly appreciate the separation of concerns this has introduced to aid in the developer experience when expanding up on what is here. This continues to be quite impressive to me!

After working a bit closely with this however, there are a few things I'd like to point out after the fact which this PR may not directly be addressing (apologies there).

• Unless that explanation has slipped me around the input for /tests/aria-at-test-run.mjs, it seems I'm unable to provide the headers for the Record Results section -> Assertions table. It also appears it's getting defaulted to empty strings when the current defaults seem to be Assertion, Success case & Failure cases. See L:220
• The deep nesting within some of the outputs such as the result of TestRun.testPage() is continues to be somewhat unsettling. For example, if I wanted, I could make a call to new TestRun({ ... }).testPage().instructions.instructions.instructions.commands.commands to retrieve the list of AT commands to be ran for the respective Test. It's somewhat excessive but I understand why it has to be like this. I do hope however that further simplification is in the future.

Other than that, general expectations of use/use cases could be further detailed to help onboard others when needed. An example of such would be in a previous meeting we had where it was explained that implementing TestWindow to then pass the resulting hooks to TestRun.

[howard-e]

@mzgoddard /tests/aria-at-test-run.mjs L:366

The default click event seems to using the incorrect event for the result.

hooks.setCommandAssertion({commandIndex, assertionIndex, result: AssertionResultMap.FAIL_MISSING}) -> hooks.setCommandAssertion({commandIndex, assertionIndex, result: AssertionResultMap.FAIL_INCORRECT})

[alflennik]

Looks good Z! I still want to look a little bit more at the code level stuff, but what I can confirm is that having new JS APIs to interact in the app is huge, and you deserve a big pat on the back for that 😜 because it was a bit difficult before to imagine wrapping the test renderer.

[alflennik]

Looks good Z! I've gotten to play around with this a lot in the revised ARIA-AT App, and it's been working great.

[howard-e]

Everything looks great to me!

The only thing that stands out to me is not having the associated label tied to the AT's Output textarea input on the test page.
At /tests/aria-at-harness.mjs:commandResult:L:334, perhaps a forInput could be added for the label and the id attached for the following textarea.

[mzgoddard]

The only thing that stands out to me is not having the associated label tied to the AT's Output textarea input on the test page.
At /tests/aria-at-harness.mjs:commandResult:L:334, perhaps a forInput could be added for the label and the id attached for the following textarea.

Noted. @howard-e Do you think it Is ok if I do that in another PR since that's in harness and this PR doesn't change the rendering behavior?

[howard-e]

Agreed. That would be more appropriate.