test: adds weekly scheduled smoke tests #28
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@veritem the https://github.com/AriPerkkio/eslint-remote-tester#eslint-remote-testerconfigjs
|
|
I haven't read about that, let me add it in another commit. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes/implements the following bugs/features
What
Adds weekly scheduled smoke tests to be run on every Sunday at 00:00. Tests can also be trigger manually (
workflow_dispatch). These tests will be looking for rule crashes.Some example ESLint community plugins that are using this tool already:
eslint-plugin-reacteslint-plugin-jesteslint-plugin-jest-extendedeslint-plugin-jest-domeslint-plugin-testing-libraryeslint-plugin-unicornHelps finding issues like below even before releasing:
no-conditional-testswhen using array destructuring #21expect-expectrule when calling function without assignment #20Here is the first run: AriPerkkio#1
How
Plugin maintainer making sure all existing rules do not crash
Utilizes https://github.com/marketplace/actions/eslint-remote-tester-runner. Each test run will check as many repositories as it can in 5 hours 30 minutes (default value of
timeLimit). Utilizes over 10K repositories from eslint-remote-tester-repositories.Available configuration options:
eslint-remote-tester-run-action: https://github.com/marketplace/actions/eslint-remote-tester-runner#action-parameterseslint-remote-tester: https://github.com/AriPerkkio/eslint-remote-tester#configuration-optionsWhen test fails the @github-actions bot will open new issue with title
Results of weekly scheduled smoke test. It will reuse the existing issue if previous one hasn't yet been closed. It's recommended to close the issues once their findings are fixed.Below are two examples of test results: