-
-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update mocha, chai for testomat reporting internal #21400
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems you have commited a couple changes to the submodules. This for sure shouldn't be included in this PR and currently causes a lot tests to fail.
You may need to revert those changes. If you need help with git or how to handle submodules feel free to get in touch with me or another dev on slack
* added testomatio reporter for ui tests * removed api key from code --------- Co-authored-by: achakko <[email protected]>
089e585
to
09f2338
Compare
Hi @sgiehl @michalkleiner I rebased to remove the commit that was not required. Thanks. |
This issue is in "needs review" but there has been no activity for 7 days. ping @matomo-org/core-reviewers |
@matomo-org/core-reviewers This reporting seems to be working now. The CI job ran tests and reported the UI test run results into testomat with a decent stack trace for failed tests as well. Runs can be accessed here. Once we have more data, will get meaningful results in Analytics. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test tomato reporting is looking great 👍
Is there any reason the UI test output log has lots of [TESTOMATIO] Warning: Test could not be matched (404) should be possible to publish new version
messages? This will make it a bit harder to check the logs, it also looks like colour highlighting has also disappeared which is helpful to quickly identify the failed tests.
…or standard spec and testomat, also added to config.
…or standard spec and testomat, also added to config.
Added color = 1 and removed the deprecated Usecolors method. Seems to be working! We have a summary of any failed just above the Error message with colors ofcourse. Good to approve? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @achakko, much easier to read now 👍
Might be good to get additional approvals for this one, as I've not been following all the other points raised.
The issue with the double reporting in testomat.io is logged here. I will follow up for resolution. testomatio/reporter#245 |
…c tests locally and testomat in CI
…o dev-17222-testomat-internal
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This overall looks good to me now. Don't have access to the testomatio dashboard, so can't say if it's working on that side.
Description:
This is a pull request to update mocha, chai version for all tests, thereby to help enable CI integration with testomat.io to understand reporting on tests including flakiness
Review