Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: run only failed tests when re-running #281

Merged
merged 26 commits into from
Jul 20, 2023
Merged

ci: run only failed tests when re-running #281

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
a3cbb6d
made it fail
RotemFB Jul 12, 2023
e6e1971
added lf flag
RotemFB Jul 12, 2023
278d0e8
deleted lf
RotemFB Jul 13, 2023
f642b08
Updated lf flag
RotemFB Jul 13, 2023
4a46f59
last failed flag
RotemFB Jul 16, 2023
6d0e0f0
Trying integration
RotemFB Jul 17, 2023
38e5d86
Added cache
RotemFB Jul 17, 2023
1634cfb
added cache dir
RotemFB Jul 17, 2023
8d4dea8
specified the cache for the pytest
RotemFB Jul 17, 2023
f7d3304
added save and restore
RotemFB Jul 17, 2023
b293c53
another try
RotemFB Jul 18, 2023
0265822
new path
RotemFB Jul 18, 2023
96bceba
updated key
RotemFB Jul 18, 2023
540fe8e
save is in run test
RotemFB Jul 18, 2023
7c447f3
new method
RotemFB Jul 18, 2023
d063450
added always condition
RotemFB Jul 18, 2023
f3d5887
another try
RotemFB Jul 18, 2023
f1748c7
cache for integration and nightly
RotemFB Jul 18, 2023
caedef6
added sha and os to the key
RotemFB Jul 18, 2023
c8634dc
fixed the cause of the errors
RotemFB Jul 18, 2023
b590a92
fixed CR comments
RotemFB Jul 19, 2023
7ea81dc
deleted spaces
RotemFB Jul 19, 2023
e2afa89
changed os to label
RotemFB Jul 20, 2023
e813e50
returned to os
RotemFB Jul 20, 2023
651d4d8
replaced os to env.RUNNER_LABEL
RotemFB Jul 20, 2023
510dfdd
os again
RotemFB Jul 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion .github/workflows/integration-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,14 @@ jobs:
api-endpoint: "api.${{ inputs.environment }}.firebolt.io"
region: "us-east-1"

- name: Restore cached failed tests
id: cache-tests-restore
uses: actions/cache/restore@v3
with:
path: |
.pytest_cache/v/cache/lastfailed
key: ${{ runner.os }}-pytest-restore-failed-${{ github.ref }}-${{ github.sha }}

- name: Run integration tests
env:
USER_NAME: ${{ env.USERNAME }}
Expand All @@ -85,8 +93,17 @@ jobs:
API_ENDPOINT: "api.${{ inputs.environment }}.firebolt.io"
ACCOUNT_NAME: "firebolt"
run: |
pytest -n 6 --dist loadgroup --timeout_method "signal" -o log_cli=true -o log_cli_level=INFO tests/integration --alluredir=allure-results
pytest --last-failed -n 6 --dist loadgroup --timeout_method "signal" -o log_cli=true -o log_cli_level=INFO tests/integration --alluredir=allure-results

- name: Save failed tests
id: cache-tests-save
uses: actions/cache/save@v3
if: failure()
with:
path: |
.pytest_cache/v/cache/lastfailed
key: ${{ steps.cache-tests-restore.outputs.cache-primary-key }}

- name: Get Allure history
uses: actions/checkout@v2
if: always()
Expand Down
19 changes: 18 additions & 1 deletion .github/workflows/nightly-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,14 @@ jobs:
region: "us-east-1"
db_suffix: ${{ format('{0}_{1}', matrix.os, matrix.python-version) }}

- name: Restore cached failed tests
id: cache-tests-restore
uses: actions/cache/restore@v3
with:
path: |
.pytest_cache/v/cache/lastfailed
key: ${{ runner.os }}-pytest-restore-failed-${{ github.ref }}-${{ github.sha }}

- name: Run integration tests
env:
USER_NAME: ${{ secrets.FIREBOLT_STG_USERNAME }}
Expand All @@ -81,8 +89,17 @@ jobs:
ACCOUNT_NAME: "firebolt"
API_ENDPOINT: "api.staging.firebolt.io"
run: |
pytest --timeout_method "thread" -o log_cli=true -o log_cli_level=INFO --junit-xml=report/junit.xml tests/integration
pytest --last-failed --timeout_method "thread" -o log_cli=true -o log_cli_level=INFO --junit-xml=report/junit.xml tests/integration

- name: Save failed tests
id: cache-tests-save
uses: actions/cache/save@v3
if: failure()
with:
path: |
.pytest_cache/v/cache/lastfailed
key: ${{ steps.cache-tests-restore.outputs.cache-primary-key }}

- name: Foresight Test / Coverage Results
uses: runforesight/foresight-test-kit-action@v1
if: success() || failure()
Expand Down