-
Notifications
You must be signed in to change notification settings - Fork 119
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
* Add MacOS functional testing in CI * Generate test addons in the binary dir * Deprecate generate_all_tests.py script * Fix cmake build path for wasm * Remove duplicate addon build during functional tests * A little bit of test renaming for consistency * Unify names and style of functional test YAML * Fix testSettings.js instability on MacOS * Fix testOnboarding.js instability on MacOS * Remove generate_all_tests.py script, it is no longer used * Fix testServers.js instability on MacOS * Try to get tests to fail faster with a condition timeout
- Loading branch information
Showing
16 changed files
with
239 additions
and
120 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| name: MacOS Tests | ||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| - "releases/**" | ||
| pull_request: | ||
| branches: | ||
| - main | ||
| - "releases/**" | ||
|
|
||
| # Restrict tests to the most recent commit. | ||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| build_test_app: | ||
| name: Build Test Client | ||
| runs-on: macos-latest | ||
| outputs: | ||
| matrix: ${{ steps.testGen.outputs.tests }} | ||
| steps: | ||
| - name: Clone repository | ||
| uses: actions/checkout@v3 | ||
| with: | ||
| submodules: 'recursive' | ||
|
|
||
| - name: Install build dependencies | ||
| shell: bash | ||
| run: | | ||
| pip3 install -r requirements.txt | ||
| brew install ninja | ||
| - name: Install Qt6 | ||
| shell: bash | ||
| run: | | ||
| wget https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/mozillavpn.v2.mozillavpn.cache.level-3.toolchains.v3.qt-mac.latest/artifacts/public%2Fbuild%2Fqt6_mac.zip -O qt6_mac.zip | ||
| unzip -a -d ${{ github.workspace }} qt6_mac.zip | ||
| - name: Compile test client | ||
| shell: bash | ||
| run: | | ||
| mkdir -p build/cmake | ||
| cmake -S $(pwd) -B build/cmake -GNinja -DCMAKE_BUILD_TYPE=RelWithDebInfo \ | ||
| -DCMAKE_PREFIX_PATH=${{ github.workspace }}/qt_dist/lib/cmake | ||
| cmake --build build/cmake -j$(nproc) --target dummyvpn | ||
| cp ./build/cmake/tests/dummyvpn/dummyvpn build/ | ||
| cp -r ./build/cmake/tests/dummyvpn/addons build/addons | ||
| - uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: test-client-${{ github.sha }} | ||
| path: | | ||
| build/ | ||
| !build/cmake/ | ||
| - name: Generate tasklist | ||
| id: testGen | ||
| shell: bash | ||
| run: | | ||
| echo -n "tests=" >> $GITHUB_OUTPUT | ||
| for test in $(find tests/functional -name 'test*.js' | sort); do | ||
| printf '{"name": "%s", "path": "%s"}' $(basename ${test%.js} | sed -n 's/test//p') $test | ||
| done | jq -s -c >> $GITHUB_OUTPUT | ||
| - name: Check tests | ||
| shell: bash | ||
| env: | ||
| TEST_LIST: ${{ steps.testGen.outputs.tests }} | ||
| run: | | ||
| echo $TEST_LIST | jq | ||
| functionaltests: | ||
| name: Functional tests | ||
| needs: | ||
| - build_test_app | ||
| runs-on: macos-latest | ||
| timeout-minutes: 45 | ||
| strategy: | ||
| fail-fast: false # Don't cancel other jobs if a test fails | ||
| matrix: | ||
| test: ${{ fromJson(needs.build_test_app.outputs.matrix) }} | ||
| steps: | ||
| - name: Clone repository | ||
| uses: actions/checkout@v3 | ||
|
|
||
| - uses: actions/download-artifact@v3 | ||
| with: | ||
| name: test-client-${{ github.sha }} | ||
| path: build/ | ||
|
|
||
| - name: Install test dependecies | ||
| run: | | ||
| pip3 install -r requirements.txt | ||
| npm install | ||
| - name: Check build | ||
| shell: bash | ||
| run: | | ||
| chmod +x ./build/dummyvpn | ||
| ./build/dummyvpn -v | ||
| - name: Running ${{ matrix.test.name }} Tests | ||
| id: runTests | ||
| uses: nick-invision/retry@v2 | ||
| with: | ||
| timeout_minutes: 15 | ||
| max_attempts: 3 | ||
| command: | | ||
| export PATH=$GECKOWEBDRIVER:$(npm bin):$PATH | ||
| export HEADLESS=yes | ||
| export TZ=Europe/London | ||
| mkdir -p $ARTIFACT_DIR | ||
| npm run functionalTest -- --retries 3 ${{ matrix.test.path }} | ||
| env: | ||
| ARTIFACT_DIR: ${{ runner.temp }}/artifacts | ||
| MVPN_BIN: ./build/dummyvpn | ||
|
|
||
| - name: Uploading artifacts | ||
| uses: actions/upload-artifact@v3 | ||
| if: ${{ always() }} | ||
| with: | ||
| name: ${{ matrix.test.name }} Logs | ||
| path: ${{ runner.temp }}/artifacts |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.