Fall 2019 Research and Prototype Roadmap

Research Roadmap

Supply feedback on this roadmap and specific items in it in Issue 9 :)

Timeline:

• Initial Research: August 19th - October 28th, 2019
• Prototype Testing System: October 28th - November 22nd, 2019

Scope:

Initial Research (August 19 - Oct 28th)

1. Design a test assertion model (How we express our expectations of correct behavior)

1. Decide on a generalized language for describing ARIA-AT test and provide a mapping between that generalized language and specific AT’s vocabulary.
   1. In progress by the ARIA-AT CG
2. Design structure test format
   1. In progress by the ARIA-AT CG
3. Author sample test material (a set of tests in the specified format)
4. Design aria-coverage analysis workflow as part of the structured test format which will help ARIA-AT CG track coverage as tests are authored.

2. Result data model

1. Decide the structure of results from tests
   1. In progress by the ARIA-AT CG
2. Sample test results for sample test material

3. Test author contributor guidelines

1. Central repository for the contribution of test material with ability to discuss and comment on tests.
2. Description of how to write and contribute test material. Criteria for acceptance of contributions, including coverage information.
3. A high level list of features that will need to be tested.
4. Guidelines on how to write a test plan for a feature (a test plan is an outline of all tests to write).

4. Tester Application

1. Application requirements and prioritization
   1. User stories in progress by ARIA AT CG https://github.com/w3c/aria-at/wiki/High-level-use-cases
2. Review of open source test management software and decide whether to move forward with one of these.
   1. Research in progress by Valerie. Tracking questions to answer in https://github.com/w3c/aria-at/issues/8
3. Draft of tester guidelines and training material.

5. Report Mockup

1. Requirements and prioritization of requirements for minimally viable report

Implement Prototype (Oct 28 - Nov 22)

1. Implement tester client
2. Implement data storage
3. Compile documentation
4. Implement report

Outstanding Questions

1. Do we want to use the APG examples as part of the test material?
   1. Pros:
      1. We can tell APG readers whether an APG example widgets (which is a specific implementation of a design pattern) is “supported”.
   2. Cons: 2. Testing APG examples only tells us AT support of that exact configuration of html and aria, which sometimes can be more complex than necessary. 3. We might want to start with a more thorough but trackable testing paradigm. Minimum aria-role/aria-attribute combinations. In this scenario, we might still be able to describe support of APG example widgets as these examples would be subsets of APG examples.
2. How structured should the results be? Discussion on going in w3c/aria-at#6
3. Resources for hosting? 3. Currently, hoping to host application from something as low cost and simple as gh-pages.
4. Who will be performing tests?