Generation of tests and review pages script improvements #451
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updated create-all-tests script to add verbose flag and provide summary of runs
* WIP - pre-push test * Revert "WIP - pre-push test" This reverts commit 42b40d6 * adding tests folder level .gitignore * after untrack * added additional command to generate-and-commit-files workflow script * Generate test and review files automatically * workflow tweak * Generate test and review files automatically * workflow tweak * Generate test and review files automatically * workflow tweak * Generate test and review files automatically * workflow tweak * Generate test and review files automatically * Generate test and review files automatically * workflow tweak * Generate test and review files automatically * workflow tweak * workflow step rename * added tests/.gitignore * Generate test and review files automatically * untracked files * Generate test and review files automatically * misc * Generate test and review files automatically * adjusted generate-and-commit-files.yml for CI * Generate test and review files automatically * updated workflow action * untracked files * Generate test and review files automatically * misc * Generate test and review files automatically * Update test-03-navigate-to-unchecked-checkbox-interaction.html * Generate test and review files automatically * Create test-03-navigate-to-unchecked-checkbox-interaction.html * Generate test and review files automatically * misc * Generate test and review files automatically * after untrack * modified gitignore * untrack files * testing .gitattributes * Generate test and review files automatically * .gitattributes * Generate test and review files automatically * misc * Generate test and review files automatically * misc * Generate test and review files automatically * misc * Generate test and review files automatically * adjusted .gitattributes * Generate test and review files automatically * misc * Generate test and review files automatically * misc * misc * adjusted gitignore * after untrack * prep to test workflow * Generate test and review files automatically * Create index.html * Generate test and review files automatically * Delete index.html * Generate test and review files automatically * adjusting generated PR link * Generate test and review files automatically * adding support for support.json * Generate test and review files automatically * build cleanups * misc * workflow refactor * Generate test and review files automatically * Generate test and review files automatically * after merging with verbosity branch * Generate test and review files automatically * updated .gitattributes and .gitignore; added additional utility script for generating tests and review files locally * Generate test and review files automatically * misc * Generate test and review files automatically Co-authored-by: howard-e <[email protected]>
* added `npm run validate` command * added `npm run cleanup` command
…all-tests.js) and npm run validate
# Conflicts: # build/index.html # build/review/checkbox-tri-state.html # build/review/checkbox.html # build/review/combobox-autocomplete-both.html # build/review/combobox-select-only.html # build/review/menu-button-actions-active-descendant.html # build/review/menubar-editor.html # build/review/modal-dialog.html # build/review/tabs-manual-activation.html
…tattributes to include *.mjs files of build/tests directory
# Conflicts: # .gitattributes # .gitignore # docs/LOCAL_DEVELOPMENT.md # index.html # package.json # review/checkbox-tri-state.html # review/checkbox.html # review/combobox-autocomplete-both.html # review/combobox-select-only.html # review/menu-button-actions-active-descendant.html # review/menubar-editor.html # review/modal-dialog.html # review/tabs-manual-activation.html # scripts/create-all-tests.js # scripts/create-example-tests.js # scripts/test-reviewer.mjs
alflennik
approved these changes
Jun 17, 2021
|
@howard-e Can we work on a branch within this repo, please? Working under your own account just means that it's more difficult for everybody else to checkout and contribute. |
|
@jesdaigle / @s3ththompson Can we get a review on this PR soon? I'd love to see it merged this week. |
jesdaigle
approved these changes
Jun 29, 2021
howard-e
added a commit
that referenced
this pull request
Jul 15, 2021
howard-e
added a commit
that referenced
this pull request
Jul 20, 2021
…or Navigation Menus (#347) * Add Disclosure for Navigation Menus APG example with modified CSS/JS references For more details of what is removed from and maintained in APG examples, see: 335dada * Example: update page title * Example: remove core JS/CSS references * Example: remove 'Related Links' nav * Example: update pattern href * Example: remove links to similar examples * Example: remove usage options and keyboard support checkbox * Example: remove 'Accessibility Features', 'Keyboard Support', 'Role, Property, State, and Tabindex Attributes' and source code sections * Example: remove footer nav * Example: add a link before the first disclosure button for use in setup scripts More details: 4357a2e * Add references.csv * Add Shift+B to keys.mjs * Correct Shift+F in keys.mjs * Add Shift+K and Shift+U to keys.mjs * Add commands.csv * Add tests.csv * Add setup scripts * Remove trailing blank lines from CSVs * Generate test and review files automatically * Example: remove blur handler which interferes with setup scripts * Generate test and review files automatically * Correct element selector for second disclosure button in setup scripts * Generate test and review files automatically * Correct document reference in setup script * Generate test and review files automatically * Apply display: none styling to the first dropdown menu for certin tests (e.g. where navigation back to the associated button is required without intermediate elements) * Generate test and review files automatically * Update setup scripts to focus the first link in a dropdown rather than the button for the next menu * Generate test and review files automatically * Add instructions about pressing certain commands twice * Generate test and review files automatically * Add additional link navigation commands when moving between disclosure button and dropdown * Generate test and review files automatically * Add VO+Command+J to VoiceOver tests when navigating forwards to the disclosure * Add Shift+VO+Command+J to VoiceOver tests when navigating backwards to the disclosure * Add Insert+Up to JAWS and NVDA tests when reading information about the disclosure * Generate test and review files automatically * Remove Insert+Up from JAWS and NVDA tests when reading information about the disclosure in interaction mode, as unlike in reading mode, only the accessible name of the control is reported * Add Home, Up and Left arrow to JAWS and NVDA tests when navigating backwards to the disclosure in interaction mode * Add Control+Option+Command+L and Shift+Control+Option+Command+L to keys.mjs file * Add VO+Command+L and Shift+VO+Command+L to VoiceOver tests when navigating to links in the dropdown * Generate test and review files automatically * Copy files to address merge conflicts * Regenerate test files * Generate test and review files automatically * after merging in new build artifact PR * Generate test and review files automatically * following re-merge of #451 * Generate test and review files automatically * Updated 'Lasted Edited' -> 'Last Edited' * Generate test and review files automatically Co-authored-by: jscholes <[email protected]> Co-authored-by: Isabel Del Castillo <[email protected]> Co-authored-by: IsaDC <[email protected]> Co-authored-by: Howard Edwards <[email protected]> Co-authored-by: howard-e <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Preview Tests - from forked repository
docs/LOCAL_DEVELOPMENT.mdfor information on new/updated script usages.npm run buildto update build folder with generated tests and review pages for all test plansnpm run validateto check if there are any validation errors for the test plansnpm run build --testplan=<test_plan>, eg.npm run build --testplan=checkboxto update build folder with a generated test and review page for a specific test plannpm run validate --validate=<test_plan>, eg.npm run validate --testplan=checkboxto check if there are any validation errors for a specific test planThis addresses a few issues identified with the automation/workflow scripts flow. See #384.
Test Plans generation script noisy output
When running
npm run create-all-teststo generate test plans, many of the messages were unnecessary when one of the main goals was only understanding if anything unexpected had happened.Verbose Output
The decision taken here was to completely silence the output except for displaying errors and providing a summary of how many tests have been generated.
A note on how many log lines have been suppressed along with instructions on how to view those messages is provided which means the original verbose output is now behind a
--verboseflag.Non-Verbose Output
Conflicts created by generated files from PR action + local development changes
npm run create-all-testsandnpm run review-testswill continue to work as normal. The only difference is that the files they generated are now stored in abuildfolder. This means means there will no longer be a chance for merge conflicts to occur on those in-line generated files when having to push/pull to/from the repository.npm run buildwhich will then be referenced in the PR'sPreview Testslink instead.basebranch (which the PR will also notify you of) before merging to avoid any possible merge conflicts but overall, the merge conflicts situation should be somewhat mitigated for the time being.