Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: migrate from test runner tap to jest #238

Closed
petermetz opened this issue Jul 24, 2020 · 6 comments
Closed

test: migrate from test runner tap to jest #238

petermetz opened this issue Jul 24, 2020 · 6 comments
Labels
backlog Tasks that are needed but we cannot prioritize them at present. bug Something isn't working dependencies Pull requests that update a dependency file good-first-issue Good for newcomers good-first-issue-200-intermediate Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. Nice-to-Have P4 Priority 4: Low Tests Anything related to tests be that automatic or manual, integration or unit, etc.
Milestone

Comments

@petermetz
Copy link
Contributor

petermetz commented Jul 24, 2020

Update 2021-08-09

This used to be about AVA but now is about Jest

Update 2020-11-02

Switched the target to AVA from jest, details below in comments.

Description

As a developer I want to be able to write unit tests for the universal packages that execute the same code both in NodeJS and in the browser so that I don't have to duplicate work in order to have test coverage for both platforms for the cross-platform packages.

While working on the stable signature generation, we've discovered that accidentally the wrong library was used initially (congratulations to Peter) which is called "tap" instead of "tape". They both do the same thing (test runners for TAP - Test Anything Protocol) but the tap library does not work in browsers while the tape library is, so to keep the build/test infrastructure of the project as simple as possible it would be the best to migrate to tape entirely since it anyway has support for TAP (the protocol)

https://www.npmjs.com/package/ava

Acceptance Criteria

  1. Complexity of build/test scripts has not increased
  2. Tests for cross-platform packages such as cactus-common can be written once and ran "anywhere" (NodeJS, browser)
  3. Developer workflow has not lengthened, did not increase in complexity (from the human perspective, not code-wise that was mentioned in 1))

cc: @takeutak @sfuji822 @hartm @jonathan-m-hamilton @AzaharaC @jordigiam @kikoncuo

@petermetz petermetz added bug Something isn't working good-first-issue Good for newcomers Nice-to-Have dependencies Pull requests that update a dependency file labels Jul 24, 2020
@petermetz
Copy link
Contributor Author

Good news discovered in the meantime: Since both tap and tape use the Test Anything Protocol under the hood, they can actually run each other's test cases without issues. E.g. you can do npx tap some-test-case.test.ts and if will "just work" regardless of the test case being written with tap or tape.
This is good news because it means we can gradually perform the migration without any sort of unstable transition required.

@petermetz
Copy link
Contributor Author

Refactoring this as per #299 (comment) to be about migrating to AVA not tape...

@petermetz petermetz changed the title chore: migrate from test runner tap to tape chore: migrate from test runner tap to AVA Nov 2, 2020
petermetz added a commit to petermetz/cacti that referenced this issue Nov 2, 2020
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 2, 2020
Fixes hyperledger-cacti#299

This does not provide a real fix for the
issue just marks the tests to be skipped
for now until we complete the migration
to a different test runner which will
implicitly provide the real fix as per
hyperledger-cacti#299 (comment) =>

Finally had the time to investigate this
properly and it is an issue that traces
back to the tap test executor unfortunately.
We already had plans to replace tap with
tape for other reasons (adding browser
testing support) but now this has
become more important than ever.

While seeing if we could use tape as the
test runner as well, I arrived at the
conclusion that we cannot because it
does not support obtaining coverage
while also compiling the Typescript code on the fly.
Another executor that does support this
and also satisfies our requirement of
being able to spit out TAP formatted
results is AVA so I will update the
relevant GH issue hyperledger-cacti#238 about test
runner migration to target AVA instead of tape...

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 2, 2020
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 6, 2020
Fixes hyperledger-cacti#299

This does not provide a real fix for the
issue just marks the tests to be skipped
for now until we complete the migration
to a different test runner which will
implicitly provide the real fix as per
hyperledger-cacti#299 (comment) =>

Finally had the time to investigate this
properly and it is an issue that traces
back to the tap test executor unfortunately.
We already had plans to replace tap with
tape for other reasons (adding browser
testing support) but now this has
become more important than ever.

While seeing if we could use tape as the
test runner as well, I arrived at the
conclusion that we cannot because it
does not support obtaining coverage
while also compiling the Typescript code on the fly.
Another executor that does support this
and also satisfies our requirement of
being able to spit out TAP formatted
results is AVA so I will update the
relevant GH issue hyperledger-cacti#238 about test
runner migration to target AVA instead of tape...

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 6, 2020
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 10, 2020
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 12, 2020
Fixes hyperledger-cacti#299

This does not provide a real fix for the
issue just marks the tests to be skipped
for now until we complete the migration
to a different test runner which will
implicitly provide the real fix as per
hyperledger-cacti#299 (comment) =>

Finally had the time to investigate this
properly and it is an issue that traces
back to the tap test executor unfortunately.
We already had plans to replace tap with
tape for other reasons (adding browser
testing support) but now this has
become more important than ever.

While seeing if we could use tape as the
test runner as well, I arrived at the
conclusion that we cannot because it
does not support obtaining coverage
while also compiling the Typescript code on the fly.
Another executor that does support this
and also satisfies our requirement of
being able to spit out TAP formatted
results is AVA so I will update the
relevant GH issue hyperledger-cacti#238 about test
runner migration to target AVA instead of tape...

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 12, 2020
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit that referenced this issue Dec 1, 2020
Fixes #299

This does not provide a real fix for the
issue just marks the tests to be skipped
for now until we complete the migration
to a different test runner which will
implicitly provide the real fix as per
#299 (comment) =>

Finally had the time to investigate this
properly and it is an issue that traces
back to the tap test executor unfortunately.
We already had plans to replace tap with
tape for other reasons (adding browser
testing support) but now this has
become more important than ever.

While seeing if we could use tape as the
test runner as well, I arrived at the
conclusion that we cannot because it
does not support obtaining coverage
while also compiling the Typescript code on the fly.
Another executor that does support this
and also satisfies our requirement of
being able to spit out TAP formatted
results is AVA so I will update the
relevant GH issue #238 about test
runner migration to target AVA instead of tape...

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Dec 4, 2020
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Dec 11, 2020
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Jan 6, 2021
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Jan 8, 2021
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Jan 8, 2021
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Jan 8, 2021
Migrate test execution to AVA from tap.
This is necessary because tap has some
unresolved issue that make the test
cases flaky and tape lacks features
regarding being able to support code
coverage and Typescript compilation
at the same time. Using AVA fixes both
of those shortcomings.

Fixes hyperledger-cacti#238

Signed-off-by: Peter Somogyvari <[email protected]>
@petermetz petermetz changed the title chore: migrate from test runner tap to AVA test: migrate from test runner tap to AVA Jun 25, 2021
@petermetz
Copy link
Contributor Author

petermetz added a commit to petermetz/cacti that referenced this issue Jun 28, 2021
WORK IN PROGRESS

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit that referenced this issue Jul 18, 2022
Migrated test from Tap to Jest

File Path:
packages/cactus-test-cmd-api-server/
src/test/typescript/integration/
remote-plugin-imports.test.ts

This is a PARTIAL resolution to issue #238

Signed-off-by: Youngone Lee <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Jul 18, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-plugin-consortium-manual/src
/test/typescript/integration/plugin-consortium-manual/get-consortium-jws-endpoint.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Jul 18, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-plugin-consortium-manual/src
/test/typescript/integration/plugin-consortium-manual/get-consortium-jws-endpoint.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Jul 18, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-plugin-consortium-manual/src
/test/typescript/integration/plugin-consortium-manual/get-consortium-jws-endpoint.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit that referenced this issue Jul 18, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-plugin-consortium-manual/src
/test/typescript/integration/plugin-consortium-manual/get-consortium-jws-endpoint.test.ts

This is a PARTIAL resolution to issue #238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Nov 22, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-plugin-ledger-connector-quorum/
src/test/typescript/integration/plugin-ledger-connector-quorum/
deploy-contract/v2.3.0-deploy-contract-from-json.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Nov 22, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-cmd-api-server/src/test/typescript/
integration/plugin-import-with-npm-install.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
petermetz referenced this issue in awadhana/cactus Nov 23, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-plugin-keychain-memory/src/
test/typescript/unit/plugin-keychain-memory.test.ts

This is a PARTIAL resolution to issue hyperledger#238

Signed-off-by: awadhana <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Nov 23, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-cmd-api-server/src/test/typescript/
integration/remote-plugin-imports.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Nov 23, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-plugin-ledger-connector-quorum/
src/test/typescript/integration/plugin-ledger-connector-quorum/
deploy-contract/v2.3.0-deploy-contract-from-json.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit that referenced this issue Nov 23, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-plugin-ledger-connector-quorum/
src/test/typescript/integration/plugin-ledger-connector-quorum/
deploy-contract/v2.3.0-deploy-contract-from-json.test.ts

This is a PARTIAL resolution to issue #238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Nov 23, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-cmd-api-server/src/test/typescript/
integration/plugin-import-with-npm-install.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Nov 23, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-cmd-api-server/src/test/typescript/
integration/plugin-import-with-npm-install.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Nov 25, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-cmd-api-server/src/test/typescript/
integration/plugin-import-with-npm-install.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit that referenced this issue Nov 25, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-test-cmd-api-server/src/test/typescript/
integration/plugin-import-with-npm-install.test.ts

This is a PARTIAL resolution to issue #238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz referenced this issue in awadhana/cactus Nov 25, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-plugin-keychain-memory/src/
test/typescript/unit/plugin-keychain-memory.test.ts

This is a PARTIAL resolution to issue hyperledger#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz referenced this issue Nov 25, 2022
Migrated test from Tap to Jest.

File Path:
packages/cactus-plugin-keychain-memory/src/
test/typescript/unit/plugin-keychain-memory.test.ts

This is a PARTIAL resolution to issue hyperledger#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to awadhana/cactus that referenced this issue Nov 26, 2022
Migrated test from Tap to Jest

File Path:
packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/
typescript/integration/plugin-htlc-eth-besu-erc20/
get-single-status-endpoint-invalid.test.ts

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit that referenced this issue Nov 26, 2022
Migrated test from Tap to Jest

File Path:
packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/
typescript/integration/plugin-htlc-eth-besu-erc20/
get-single-status-endpoint-invalid.test.ts

This is a PARTIAL resolution to issue #238

Signed-off-by: awadhana <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to Leeyoungone/cactus that referenced this issue Nov 29, 2022
File Path:
packages/cactus-test-
tooling/src/test/
typescript/integration/
common/
containers.test.ts

Edit from Peter: I updated the ci.yml file as well (this became a
requirement some time after this PR was opened & reviewed).

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: Youngone Lee <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit to Leeyoungone/cactus that referenced this issue Nov 29, 2022
File Path:
packages/cactus-test-
tooling/src/test/
typescript/integration/
common/
containers.test.ts

Edit from Peter: I updated the ci.yml file as well (this became a
requirement some time after this PR was opened & reviewed).

This is a PARTIAL resolution to issue hyperledger-cacti#238

Signed-off-by: Youngone Lee <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
petermetz pushed a commit that referenced this issue Nov 29, 2022
File Path:
packages/cactus-test-
tooling/src/test/
typescript/integration/
common/
containers.test.ts

Edit from Peter: I updated the ci.yml file as well (this became a
requirement some time after this PR was opened & reviewed).

This is a PARTIAL resolution to issue #238

Signed-off-by: Youngone Lee <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
@petermetz petermetz added Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. backlog Tasks that are needed but we cannot prioritize them at present. labels Aug 22, 2023
@petermetz petermetz added this to the v3.0.0 milestone Aug 22, 2023
@petermetz petermetz added P4 Priority 4: Low Tests Anything related to tests be that automatic or manual, integration or unit, etc. labels Sep 7, 2023
@petermetz
Copy link
Contributor Author

Closing this because we've moved on to managing the migration through individual tasks per test case or small group of test cases instead of one big epic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog Tasks that are needed but we cannot prioritize them at present. bug Something isn't working dependencies Pull requests that update a dependency file good-first-issue Good for newcomers good-first-issue-200-intermediate Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. Nice-to-Have P4 Priority 4: Low Tests Anything related to tests be that automatic or manual, integration or unit, etc.
Projects
None yet
2 participants