-
Notifications
You must be signed in to change notification settings - Fork 190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Axe results violations save #498
base: master
Are you sure you want to change the base?
Conversation
…ations method and user message display when new snapshot created
… into axe-results-violations-save
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- I don't think the failed violations are getting saved in the log file. Can you double check it?
- Would we be updating the same violation file everytime?
- I remember discussing, we wanted few lines of violations printed in log and the whole violations in the text file. Currently I just see a message
+------------------------------------------------+
| Tests Failed |
+------------------------------------------------+
| test_accessibility: |
| ✗ Accessibility checks for contact page failed |
| |
+------------------------------------------------+
How would the user know what's the error and where he can look for the violation?
May not be required for this PR, we need to figure out an easy way for user to accept the violation and add it to a snapshot in case they are ok with the new violation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also i feel lot of complexity is in the test.
The initialize snapshot can be a separate method and moved to utility
Violation comparison can also move to the utility
@avinash010 looking at your output below, it looks like no new violations found to save in the log file
Yes, as of now if the violation txt file exist in the conf then its gonna update the same file.
Looking at this output there are no new violations caught when compared with a snapshot file, if new violations are caught then it should print the first 50 characters from each of the violations.
Along with the violations first 50 characters it will display the conf path in the output . Could you please retry running the tests by doing a pre-request to change the HTML element contents in the existing snapshot_output_contact_page.json file which is saved under ./conf/snapshot/snahshot_output_contact_page.json |
I have moved these methods to snapshot_utils.py @avinash010 |
@avinash010 I havent pushed the latest commit yet, I will push by tomorrow. please hold of the review process. I will post here as soon the pr is ready |
To run the pytest from root dir:
python -m pytest tests/test_accessibility.py
expected Output when running first time:
| PASS: No existing snapshot was found for page. A new snapshot has been created. Please review the snapshot for violations before running the test again.