Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add testing groups and browser testing environment #728

Closed
wants to merge 52 commits into from
Closed
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
9ee8d19
add `jest-runner-groups` and tag tests
Dhaiwat10 Jan 13, 2023
28fbf75
add `jsdom` and new workflow file to test
Dhaiwat10 Jan 13, 2023
df2e508
fix typo in workflow file
Dhaiwat10 Jan 13, 2023
5ccf2b6
Revert "fix typo in workflow file"
Dhaiwat10 Jan 13, 2023
8799f16
fix usage of `pnpm test` in CI
Dhaiwat10 Jan 13, 2023
4abf41b
run all 4 test groups in parallel
Dhaiwat10 Jan 13, 2023
b647cca
rename workflow files
Dhaiwat10 Jan 13, 2023
5aa382a
fix some misgrouped tests
Dhaiwat10 Jan 16, 2023
8e50548
run services for e2e tests
Dhaiwat10 Jan 16, 2023
63d54a8
add empty changeset
Dhaiwat10 Jan 16, 2023
51d4bb4
polyfill `TextEncoder` and `TextDecoder`
Dhaiwat10 Jan 16, 2023
f4e8481
Merge branch 'master' into dp/testing-groups
Dhaiwat10 Jan 17, 2023
b790b7b
Merge branch 'master' into dp/testing-groups
Dhaiwat10 Jan 31, 2023
4bc2268
use puppeteer for browser unit tests
Dhaiwat10 Jan 31, 2023
35faf15
use pupeteer for all browser tests, remove jsdom
Dhaiwat10 Jan 31, 2023
7677df5
delete old test workflow, move lint checks
Dhaiwat10 Jan 31, 2023
79b3bb0
add build step to lint action
Dhaiwat10 Jan 31, 2023
1aa45ce
run e2e tests sequentially
Dhaiwat10 Jan 31, 2023
df94fce
Merge branch 'master' into dp/testing-groups
Dhaiwat10 Feb 3, 2023
9127f1c
Merge branch 'master' into dp/testing-groups
Dhaiwat10 Feb 24, 2023
a49de06
add label to `account.test.ts`
Dhaiwat10 Feb 24, 2023
c3338b6
try merging coverage reports using gh artifacts
Dhaiwat10 Feb 24, 2023
05d748b
temp remove node e2e tests from reqd list bc theyre failing
Dhaiwat10 Feb 24, 2023
75665c1
Update merge-coverage.yaml
Dhaiwat10 Feb 24, 2023
ca11124
Update merge-coverage.yaml
Dhaiwat10 Feb 24, 2023
1b1e7e2
rename workflows
Dhaiwat10 Feb 25, 2023
6e9f289
Update merge-coverage.yaml
Dhaiwat10 Feb 25, 2023
b49503d
update logic
Dhaiwat10 Feb 25, 2023
3e0262b
Update merge-coverage.yaml
Dhaiwat10 Feb 25, 2023
db9e535
Update merge-coverage.yaml
Dhaiwat10 Feb 25, 2023
a37da5d
Update merge-coverage.yaml
Dhaiwat10 Feb 25, 2023
3d0d4b6
Update merge-coverage.yaml
Dhaiwat10 Feb 27, 2023
9e434b0
Update merge-coverage.yaml
Dhaiwat10 Feb 27, 2023
44d5c29
update workflow files
Dhaiwat10 Feb 27, 2023
a60c542
Update merge-coverage.yaml
Dhaiwat10 Feb 27, 2023
3cf3d9b
rename workflow file
Dhaiwat10 Feb 27, 2023
e96513b
run workflows only for this pr
Dhaiwat10 Feb 27, 2023
3549d51
only run workflow for this pr
Dhaiwat10 Feb 27, 2023
ab2f3b6
only run workflow for this pr
Dhaiwat10 Feb 27, 2023
3bc6561
only run workflow for this pr
Dhaiwat10 Feb 27, 2023
9058236
only run workflow for this pr
Dhaiwat10 Feb 27, 2023
131fbcd
Update browser-e2e-tests.yaml
Dhaiwat10 Feb 27, 2023
7a3e7ff
fix
Dhaiwat10 Feb 27, 2023
8b460f4
reset pr-lint workflow
Dhaiwat10 Feb 27, 2023
25f416c
Merge branch 'master' into dp/testing-groups
Dhaiwat10 Feb 27, 2023
bd5354e
remove jest-runner-groups
Dhaiwat10 Mar 6, 2023
863f636
rename test file
Dhaiwat10 Mar 6, 2023
4f32fa5
remove `--group`
Dhaiwat10 Mar 6, 2023
1a79736
change patterns
Dhaiwat10 Mar 6, 2023
0b1086a
try `npx jest` instead of `pnpm test`
Dhaiwat10 Mar 6, 2023
7ac4759
update patterns
Dhaiwat10 Mar 6, 2023
5e7427c
Update sumMerkleTree.common.unit.test.ts
Dhaiwat10 Mar 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .changeset/spicy-llamas-fetch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
---
Empty file.
44 changes: 44 additions & 0 deletions .github/workflows/browser-e2e-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: "Browser e2e tests"

on:
pull_request:
push:
branches:
- master

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
browser-e2e-tests:
# only run this job if the base branch is dp/testing-groups
if: ${{github.ref_name == '728/merge'}}
runs-on: ubuntu-latest
outputs:
status: ${{ job.status }}
steps:
- name: Checkout
uses: actions/checkout@v3

- name: CI Setup
uses: ./.github/actions/ci-setup

- name: Build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These steps are repeated and should all be abstracted away into composite actions, such as a wrapper like this:
https://github.com/FuelLabs/fuels-ts/blob/master/.github/actions/ci-setup/action.yaml

That would apply to all of these "setup" steps as well as the "coverage" and "teardown" steps after.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also have you looked at using the advanced GitHub action called a test matrix?

Alongside the test matrix idea, you could also investigate fail-fast or the similar continue-on-error to let builds find the bad linting first, then bad unit tests, then get into the meaty stuff like e2e

run: |
BUILD_VERSION="0.0.0-${{ github.ref_name }}-$(git rev-parse --short $GITHUB_SHA)" pnpm build
- name: Run services
run: pnpm services:run

- name: Browser e2e tests
run: npx jest -- "**/*.common.e2e.test.ts" --setupFilesAfterEnv=./setup-puppeteer.ts --testTimeout=30000 --coverage

- name: Upload coverage to Github artifacts
uses: actions/upload-artifact@v2
with:
name: browser-e2e-coverage
path: coverage

- name: Stop services
run: pnpm services:clean
38 changes: 38 additions & 0 deletions .github/workflows/browser-unit-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: "Browser unit tests"

on:
pull_request:
push:
branches:
- master

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
browser-unit-tests:
# only run this job if the base branch is dp/testing-groups
if: ${{github.ref_name == '728/merge'}}
runs-on: ubuntu-latest
outputs:
status: ${{ job.status }}
steps:
- name: Checkout
uses: actions/checkout@v3

- name: CI Setup
uses: ./.github/actions/ci-setup

- name: Build
run: |
BUILD_VERSION="0.0.0-${{ github.ref_name }}-$(git rev-parse --short $GITHUB_SHA)" pnpm build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as mentioned, still need to at very least move these into an action

- name: Browser unit tests
run: npx jest -- "**/*.common.unit.test.ts" --setupFilesAfterEnv=./setup-puppeteer.ts --testTimeout=30000 --coverage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This just launches puppeteer and then runs the tests on node?


- name: Upload coverage to Github artifacts
uses: actions/upload-artifact@v2
with:
name: browser-unit-coverage
path: coverage
44 changes: 44 additions & 0 deletions .github/workflows/node-e2e-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: "Node e2e tests"

on:
pull_request:
push:
branches:
- master

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
node-e2e-tests:
# only run this job if the base branch is dp/testing-groups
if: ${{github.ref_name == '728/merge'}}
Copy link
Contributor

@camsjams camsjams Mar 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can remove this check from all the files as this action only exists on your PR anyways, unless you had noticed some kind of weird side effect

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are having to keep this here because we will need to merge this PR in before being fully done with this whole setup, as things stand. Please see: #728 (comment)

runs-on: ubuntu-latest
outputs:
status: ${{ job.status }}
steps:
- name: Checkout
uses: actions/checkout@v3

- name: CI Setup
uses: ./.github/actions/ci-setup

- name: Build
run: |
BUILD_VERSION="0.0.0-${{ github.ref_name }}-$(git rev-parse --short $GITHUB_SHA)" pnpm build
- name: Run services
run: pnpm services:run

- name: Node e2e tests
run: npx jest -- "**/*.node.e2e.test.ts" --coverage

- name: Upload coverage to Github artifacts
uses: actions/upload-artifact@v2
with:
name: node-e2e-coverage
path: coverage

- name: Stop services
run: pnpm services:clean
38 changes: 38 additions & 0 deletions .github/workflows/node-unit-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: "Node unit tests"

on:
pull_request:
push:
branches:
- master

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
node-unit-tests:
# only run this job if the base branch is dp/testing-groups
if: ${{github.ref_name == '728/merge'}}
runs-on: ubuntu-latest
outputs:
status: ${{ job.status }}
steps:
- name: Checkout
uses: actions/checkout@v3

- name: CI Setup
uses: ./.github/actions/ci-setup

- name: Build
run: |
BUILD_VERSION="0.0.0-${{ github.ref_name }}-$(git rev-parse --short $GITHUB_SHA)" pnpm build
- name: Node unit tests
run: npx jest -- "**/*.node.unit.test.ts" --coverage

- name: Upload coverage to Github artifacts
uses: actions/upload-artifact@v2
with:
name: node-unit-coverage
path: coverage
51 changes: 51 additions & 0 deletions .github/workflows/test-coverage.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: "Test coverage"

on:
workflow_run:
workflows:
[Node unit tests, Node e2e tests, Browser unit tests, Browser e2e tests]
types: [completed]

jobs:
collect-coverage:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Download coverage report for Browser unit tests
uses: dawidd6/action-download-artifact@v2
with:
workflow: browser-unit-tests.yaml
name: browswer-unit-coverage
path: browser-unit-coverage

- name: Download coverage report for Browser e2e tests
uses: dawidd6/action-download-artifact@v2
with:
workflow: browser-e2e-tests.yaml
name: browswer-e2e-coverage
path: browser-e2e-coverage

- name: Download coverage report for Node unit tests
uses: dawidd6/action-download-artifact@v2
with:
workflow: node-unit-tests.yaml
name: node-unit-coverage
path: node-unit-coverage

- name: Download coverage report for Node e2e tests
uses: dawidd6/action-download-artifact@v2
with:
workflow: node-e2e-tests.yaml
name: node-e2e-coverage
path: node-e2e-coverage

- name: Merge all coverage reports
run: npx istanbul-merge --out coverage-merged.json browser-unit-coverage/coverage-final.json browser-e2e-coverage/coverage-final.json node-unit-coverage/coverage-final.json node-e2e-coverage/coverage-final.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!


- name: Upload coverage report to Codecov
uses: codecov/codecov-action@v3
with:
files: coverage-merged.json
token: ${{ secrets.CODECOV_TOKEN }}
2 changes: 0 additions & 2 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -26,11 +26,9 @@ jobs:
- name: Build
run: |
BUILD_VERSION="0.0.0-${{ github.ref_name }}-$(git rev-parse --short $GITHUB_SHA)" pnpm build
- name: Lint
run: |
pnpm lint
- name: Checking PR Number
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the Run tests and publish coverage step below be removed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we merge this PR, we still want the rest of the repo to use this old test action instead of the new one. More context here: #728 (comment)

That's why I have kept it intact still.

uses: jwalton/gh-find-current-pr@v1
id: findPr
5 changes: 5 additions & 0 deletions jest.env.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
import './hbs.d';

import { hexlify } from '@ethersproject/bytes';
import { TextEncoder, TextDecoder } from 'util';

import faucets from './services/fuel-core/faucets.json';

global.TextEncoder = TextEncoder;
// @ts-expect-error - incompatible types
global.TextDecoder = TextDecoder;

/**
* Gets the private key for the current Jest worker.
*
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -76,6 +76,7 @@
"npm-run-all": "^4.1.5",
"open": "^8.4.0",
"prettier": "^2.7.1",
"puppeteer": "^19.6.2",
"shelljs": "^0.8.5",
"ts-generator": "^0.1.1",
"ts-jest": "^29.0.3",
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -10,6 +10,9 @@ const contractBytecode = readFileSync(
join(__dirname, '../test-projects/generic-types-contract/out/debug/generic-types-contract.bin')
);

/*
* @group node/e2e
*/
describe('GenericTypesContract', () => {
it('should call complex contract function with generic type', async () => {
const contract = await setup({
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
165 changes: 165 additions & 0 deletions pnpm-lock.yaml
12 changes: 12 additions & 0 deletions setup-puppeteer.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import type { Browser } from 'puppeteer';
import puppeteer from 'puppeteer';

let browser: Browser;

beforeAll(async () => {
browser = await puppeteer.launch();
});

afterAll(async () => {
await browser.close();
});