Zombienet add tests #1321

pepoviola · 2022-05-31T16:42:10Z

Hi @skunert, this add the missing integration test to zombienet_test and rename the command of the previous. Also, there is an small change in the test binary to allow to load the chain-spec from json because is needed to create a different genesis-state in the migrate_solo_to_para test.

NOTE: The ci configuration isn't included here, tracked in this issue: https://github.com/paritytech/ci_cd/issues/436

Thanks!

pepoviola · 2022-05-31T16:43:17Z

zombienet_tests/0004-runtime_upgrade.feature

+dave: is up
+alice: parachain 2000 is registered within 225 seconds
+charlie: reports block height is at least 5 within 250 seconds
+charlie: parachain 2000 perform upgrade with <wasm_binary_spec_version_incremented.rs.compact.compressed.wasm> within 200 seconds


NOTE: We need to have access to this artifact locally or through http to make the upgrade.

How does this work in practice? We upload the to-be-upgraded runtime somewhere and enter a url here which would befetches?

zombienet_tests/0005-migrate_solo_to_para.toml

bkchr · 2022-06-01T07:22:25Z

zombienet_tests/0004-runtime_upgrade.toml

+
+  [[relaychain.nodes]]
+  name = "bob"
+  image = "docker.io/paritypr/polkadot-debug:5236-0.9.18-c55660e9-be16bd72"


Why do we need this image here?

Hi @bkchr, no this image should be set as env var by the ci job. The ci isn't ready yet, I will update this in all the files.

Thanks!

skunert · 2022-06-05T11:00:06Z

This looks good to me. @pepoviola title says WIP, what needs to be done here?
@bkchr after this is merged and CI is working, we want to completely remove the old integration tests, correct?

pepoviola · 2022-06-06T10:48:05Z

This looks good to me. @pepoviola title says WIP, what needs to be done here? @bkchr after this is merged and CI is working, we want to completely remove the old integration tests, correct?

Hi @skunert, yes. I will need to make a couple of changes and also add the test to .gitlab-ci.yaml.
@bkchr is ok to use the image polkadot-debug:master for the relay chain nodes?

Thanks!

bkchr · 2022-06-07T09:12:02Z

@bkchr is ok to use the image polkadot-debug:master for the relay chain nodes?

IDK. What does this entails?

alvicsam

CI part looks good

pepoviola · 2022-06-07T16:07:37Z

@bkchr is ok to use the image polkadot-debug:master for the relay chain nodes?

IDK. What does this entails?

That we always use the latest image from master for polkadot. Maybe make more sense to use the latest released version, but we should add more logic to check that and not update manually the test spec.

Thanks!

pepoviola · 2022-07-02T17:55:50Z

Hi @skunert, all tests are now migrated and working on gitlab.

0001-sync_blocks_from_tip_without_connected_collator
0002-pov_recovery
0003-full_node_catching_up
0004-runtime_upgrade
0005-migrate_solo_to_para

Let me know if you want to add more tests.

Thanks!

bkchr · 2022-08-08T18:48:46Z

@pepoviola sounds really good! So, we can merge this now?

pepoviola · 2022-08-08T18:51:01Z

@pepoviola sounds really good! So, we can merge this now?

Hi @bkchr, thanks! Yes, we can merge now :)

skunert · 2022-08-09T07:18:28Z

Nice work here 👍.
One thing though, I compared all the tests again and noticed that none of the zombienet tests test the RPC functionality.
For example, the original test

cumulus/test/service/tests/full_node_catching_up.rs

Lines 83 to 84 in f7e5d70

    
           .use_external_relay_chain_node_at_port(ws_port) 
        
           .build()

connects one node via RPC, this does not happen in the zombienet tests.

Missing for tests

001
002
003

If this is a technical limitation for now, we could merge nevertheless but can not deprecate the rust tests until we have this. As I understand both zombienet and the old ones would run on CI for now?

pepoviola · 2022-08-09T08:56:43Z

Nice work here 👍. One thing though, I compared all the tests again and noticed that none of the zombienet tests test the RPC functionality. For example, the original test

cumulus/test/service/tests/full_node_catching_up.rs

Lines 83 to 84 in f7e5d70

.use_external_relay_chain_node_at_port(ws_port)

.build()

connects one node via RPC, this does not happen in the zombienet tests.
Missing for tests

001

002

003

If this is a technical limitation for now, we could merge nevertheless but can not deprecate the rust tests until we have this. As I understand both zombienet and the old ones would run on CI for now?

Hi @skunert, thanks for the feedback!. Those test (0001, 0002, 0003) check if the parachain 2000 is registered through the query interface using polakadotjs api.

const parachains = (await api.query.paras.parachains<Vec<ParaId>>()) || [];

If you want we can add one extra rcp call like api.rpc.system.chain()

Thanks!

skunert · 2022-08-09T10:07:48Z

Hey @pepoviola,
checking that the parachain is registered is not the problem here.

In test 0003, in the original rust test we start node eve as full node with cli argument --relay-chain-rpc-url <url_to_alice>:<alice_ws_port> which tests the rpc functionality. So this is what we would need to do in zombienet too.

pepoviola · 2022-08-09T10:52:26Z

Hey @pepoviola, checking that the parachain is registered is not the problem here.

In test 0003, in the original rust test we start node eve as full node with cli argument --relay-chain-rpc-url <url_to_alice>:<alice_ws_port> which tests the rpc functionality. So this is what we would need to do in zombienet too.

Ahh... my bad @skunert. Yes, we can modify the network config to test the rpc with that cli argument. I will change that and ping you again.

Thanks!!

* Trivial networking changes for Substrate PR paritytech/substrate#11940 * Apply formatting rules * update lockfile for {"polkadot", "substrate"} Co-authored-by: parity-processbot <>

pepoviola · 2022-08-10T18:38:56Z

Hi @skunert, I add the logic to use relay-chain-rpc-url in zombienet and I updated the tests.

Thanks!

skunert

LGTM!

pepoviola · 2022-08-18T13:36:54Z

Ping @skunert we need one more review to merge this one :)

pepoviola added 4 commits May 31, 2022 07:12

changes to read json spec in test binary

09bf90d

add zombienet tests

dcad8ca

fmt

bdfdaa7

Merge branch 'master' into zombienet-add-tests

2345378

pepoviola requested a review from skunert May 31, 2022 16:42

pepoviola commented May 31, 2022

View reviewed changes

pepoviola added A0-pleasereview B0-silent Changes should not be mentioned in any release notes labels May 31, 2022

bkchr reviewed Jun 1, 2022

View reviewed changes

use {{COL_IMAGE}} and clean config

7c7d767

pepoviola added the C1-low 📌 label Jun 1, 2022

pepoviola added 2 commits June 6, 2022 10:52

add comment and use relay image from env

a259d96

Merge branch 'master' into zombienet-add-tests

a52d2e2

use test-parachain image from pr

0eb64ed

pepoviola requested a review from a team as a code owner June 7, 2022 09:09

alvicsam approved these changes Jun 7, 2022

View reviewed changes

paritytech-ci requested a review from a team June 7, 2022 15:08

pepoviola added 8 commits June 14, 2022 07:37

Merge branch 'master' into zombienet-add-tests

a9c8d6f

fix warns

47adb87

fix warns

c13240a

fmt

e6a0271

typo

20119d7

fix ci to use zombienet image

1c62663

fix spawn nodes for test

4681a2c

reorg test

f22668f

pepoviola changed the title ~~[wip] Zombienet add tests~~ Zombienet add tests Jul 2, 2022

Merge branch 'master' into zombienet-add-tests

f7e5d70

sergejparity approved these changes Aug 9, 2022

View reviewed changes

skunert and others added 6 commits August 10, 2022 12:04

Collator key only needed if we run as collator

d157125

[Fix] Benchmark build artifact folder creation (#1518)

2c85907

Trivial networking changes for Substrate PR #11940 (#1486)

c8a6fe0

* Trivial networking changes for Substrate PR paritytech/substrate#11940 * Apply formatting rules * update lockfile for {"polkadot", "substrate"} Co-authored-by: parity-processbot <>

bump zombienet version

c604c2c

update network def for test

25a1b00

typo

053ec80

merge master into feature branch

a994124

skunert approved these changes Aug 11, 2022

View reviewed changes

the-right-joyce added A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders. I5-tests Tests need fixing, improving or augmenting. and removed A0-pleasereview labels Aug 12, 2022

bkchr approved these changes Aug 18, 2022

View reviewed changes

bkchr merged commit 725d007 into master Aug 18, 2022

bkchr deleted the zombienet-add-tests branch August 18, 2022 15:59

librelois mentioned this pull request Sep 15, 2022

Update substrate/polkadot/cumulus from v0.9.26 to v0.9.29 moonbeam-foundation/moonbeam#1814

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zombienet add tests #1321

Zombienet add tests #1321

pepoviola commented May 31, 2022

pepoviola May 31, 2022

skunert Jun 5, 2022

bkchr Jun 1, 2022

pepoviola Jun 1, 2022

skunert commented Jun 5, 2022

pepoviola commented Jun 6, 2022

bkchr commented Jun 7, 2022

alvicsam left a comment

pepoviola commented Jun 7, 2022

pepoviola commented Jul 2, 2022

bkchr commented Aug 8, 2022

pepoviola commented Aug 8, 2022

skunert commented Aug 9, 2022

pepoviola commented Aug 9, 2022

skunert commented Aug 9, 2022

pepoviola commented Aug 9, 2022

pepoviola commented Aug 10, 2022

skunert left a comment

pepoviola commented Aug 18, 2022

Zombienet add tests #1321

Zombienet add tests #1321

Conversation

pepoviola commented May 31, 2022

pepoviola May 31, 2022

Choose a reason for hiding this comment

skunert Jun 5, 2022

Choose a reason for hiding this comment

bkchr Jun 1, 2022

Choose a reason for hiding this comment

pepoviola Jun 1, 2022

Choose a reason for hiding this comment

skunert commented Jun 5, 2022

pepoviola commented Jun 6, 2022

bkchr commented Jun 7, 2022

alvicsam left a comment

Choose a reason for hiding this comment

pepoviola commented Jun 7, 2022

pepoviola commented Jul 2, 2022

bkchr commented Aug 8, 2022

pepoviola commented Aug 8, 2022

skunert commented Aug 9, 2022

pepoviola commented Aug 9, 2022

skunert commented Aug 9, 2022

pepoviola commented Aug 9, 2022

pepoviola commented Aug 10, 2022

skunert left a comment

Choose a reason for hiding this comment

pepoviola commented Aug 18, 2022