Add development chain-spec file for minimal/parachain templates for Omni Node compatibility #6529

EleisonC · 2024-11-18T18:45:05Z

Description

This PR adds development chain specs for the minimal and parachain templates. #6334

Integration

This PR adds development chain specs for the minimal and para chain template runtimes, ensuring synchronization with runtime code. It updates zombienet-omni-node.toml, zombinet.toml files to include valid chain spec paths, simplifying configuration for zombienet in the parachain and minimal template.

Review Notes

Overview of Changes:

Added development chain specs for use in the minimal and parachain template.
Updated zombienet-omni-node.toml and zombinet.toml files in the minimal and parachain templates to include paths to the new dev chain specs.

Integration Guidance:
NB: Follow the templates' READMEs from the polkadot-SDK master branch. Please build the binaries and runtimes based on the polkadot-SDK master branch.

Ensure you have set up your runtimes parachain-template-runtime and minimal-template-runtime
Ensure you have installed the nodes required ie parachain-template-node and minimal-template-node
Set up Zombinet
For running the parachains, you will need to install the polkadot cargo install --path polkadot remember from the polkadot-SDK master branch.
Inside the template folders minimal or parachain, run the command to start with Zombienet with Omni Node, Zombienet with minimal-template-node or Zombienet with parachain-template-node

Include your leftover TODOs, if any, here.

Test the syncing of chain specs with runtime's code.

cla-bot-2021 · 2024-11-18T18:45:12Z

User @EleisonC, please sign the CLA here.

…adot-sdk into add-dev-chain-spec

This reverts commit 80669c1.

iulianbarbu

Thanks @EleisonC ! Left a few comments for now.
Looking forward to the README updates & btw, have you clarified how to make a rust test for the chain specs?

iulianbarbu · 2024-11-20T13:40:47Z

templates/minimal/zombienet.toml

@@ -4,6 +4,7 @@
 [relaychain]
 chain = "dev"
 default_command = "minimal-template-node"
+chain_spec_path = "./dev_chain_spec.json"


This isn't needed here. minimal-template-node loads the chain spec based on the wasm binary blob, within the node itself.

iulianbarbu · 2024-11-20T13:41:11Z

templates/parachain/zombienet.toml

@@ -14,6 +14,7 @@ ws_port = 9955

 [[parachains]]
 id = 1000
+chain_spec_path = "./dev_chain_spec.json"


Not needed here either.

EleisonC · 2024-11-20T13:55:29Z

Thanks @EleisonC ! Left a few comments for now. Looking forward to the README updates & btw, have you clarified how to make a rust test for the chain specs?

Hey @iulianbarbu that is what I am looking into today. Do you have any pointers you'd like me to know as I move forward?

iulianbarbu · 2024-11-20T14:06:34Z

Thanks @EleisonC ! Left a few comments for now. Looking forward to the README updates & btw, have you clarified how to make a rust test for the chain specs?

Hey @iulianbarbu that is what I am looking into today. Do you have any pointers you'd like me to know as I move forward?

From the top of my head I thought about adding an integration test. You could build a ChainSpec as described here: https://github.com/paritytech/polkadot-sdk/blob/master/templates/parachain/node/src/chain_spec.rs#L36, and then call as_json -> and then convert it to serde_json::Value, and then read the newly added file and load its content as a serde_json::Value and then compare the two.

EleisonC · 2024-11-21T06:10:50Z

Hey @iulianbarbu one last question it's okay if I placed the test folder/file under the nodes folder/file. if not is there a better place

iulianbarbu · 2024-11-21T09:37:35Z

Hey @iulianbarbu one last question it's okay if I placed the test folder/file under the nodes folder/file. if not is there a better place

Hmm, thought about placing it under runtime initially. I think it makes more sense to place it under runtime so that if anything changes here, running the tests for the crate would fail because the changes will not be inline with dev_chain_spec.json. The node uses the logic defined in the runtime, so it would benefit from the runtime test, but not directly. I don't think we need to add a test on the same lines in the templates' nodes crate either because these nodes aren't depending on the dev_chain_spec.json file. Only OmniNode depends on this file being in sync with the runtime, and that not because it would fail otherwise, but because it would start an outdated version.

EleisonC · 2024-11-25T14:19:51Z

From the top of my head I thought about adding an integration test. You could build a ChainSpec as described here: https://github.com/paritytech/polkadot-sdk/blob/master/templates/parachain/node/src/chain_spec.rs#L36, and then call as_json -> and then convert it to serde_json::Value, and then read the newly added file and load its content as a serde_json::Value and then compare the two.

Hey @iulianbarbu regarding comparing the two I am a bit stuck on what to compare. From my research I would think it's other the code or value of top
Is it the value of code? Because the test fails when I compare the code value of the two chain spec files.
let test_runtime_code = test_json_value // .get("genesis") // .expect("failed to extract genesis") // .get("runtimeGenesis") // .expect("failed to extract runtimeGenesis from genesis") // .get("code") // .expect("failed to extract code from runtimeGenesis");
But the code value of test_runtime and the existing file do not match. Hence wondering if it would be the Top value to or maybe I am missing a step of conversion somewhere

EleisonC · 2024-11-26T02:54:35Z

From the top of my head I thought about adding an integration test. You could build a ChainSpec as described here: https://github.com/paritytech/polkadot-sdk/blob/master/templates/parachain/node/src/chain_spec.rs#L36, and then call as_json -> and then convert it to serde_json::Value, and then read the newly added file and load its content as a serde_json::Value and then compare the two.

Hey @iulianbarbu regarding comparing the two I am a bit stuck on what to compare. From my research I would think it's other the code or value of top Is it the value of code? Because the test fails when I compare the code value of the two chain spec files. let test_runtime_code = test_json_value // .get("genesis") // .expect("failed to extract genesis") // .get("runtimeGenesis") // .expect("failed to extract runtimeGenesis from genesis") // .get("code") // .expect("failed to extract code from runtimeGenesis"); But the code value of test_runtime and the existing file do not match. Hence wondering if it would be the Top value to or maybe I am missing a step of conversion somewhere

I figured out my problem was around decoding a Hex String to bytes. I can now properly traverse the values of the Top section and find the code for that spec file

iulianbarbu · 2024-11-27T12:45:36Z

From the top of my head I thought about adding an integration test. You could build a ChainSpec as described here: https://github.com/paritytech/polkadot-sdk/blob/master/templates/parachain/node/src/chain_spec.rs#L36, and then call as_json -> and then convert it to serde_json::Value, and then read the newly added file an

d load its content as a serde_json::Value and then compare the two.

Hey @iulianbarbu regarding comparing the two I am a bit stuck on what to compare. From my research I would think it's other the code or value of top Is it the value of code? Because the test fails when I compare the code value of the two chain spec files. let test_runtime_code = test_json_value // .get("genesis") // .expect("failed to extract genesis") // .get("runtimeGenesis") // .expect("failed to extract runtimeGenesis from genesis") // .get("code") // .expect("failed to extract code from runtimeGenesis"); But the code value of test_runtime and the existing file do not match. Hence wondering if it would be the Top value to or maybe I am missing a step of conversion somewhere

I figured out my problem was around decoding a Hex String to bytes. I can now properly traverse the values of the Top section and find the code for that spec file

Happy to hear you've sorted it out! I've noticed that in some tests we don't compare the code portions of the generated chain specs, and that's mainly because they are not super relevant for the test case. For this test though I think that it is relevant, to ensure generated chain specs correspond to the runtimes in the CI.

Some other good news is that once we'll have these chain specs we'll be able to easily run in the CI the omni_node_* tests like this one: https://github.com/paritytech/polkadot-sdk/blob/master/templates/zombienet/tests/smoke.rs#L160. We can update them in a follow up, no need to consider them right now, but it is up to you.

As usual, LMK in case of blockers, happy to take a look.

Signed-off-by: EleisonC <[email protected]>

EleisonC · 2024-11-27T22:47:09Z

Hey @iulianbarbu,
I added a test and would appreciate it if you could look at it and approve it before I add the one for the para chain. Additionally, I added the test folder to nodes because I was encountering setup issues.

iulianbarbu

Can you please regenerate the dev_chain_spec.json for minimal (for now) after building the minimal/runtime with cargo build? I wonder if you built the runtime with --release and then generated the dev_chain_spec.json. It makes sense according to the README, but we run the tests with the testnet profile, which enables the debug symbols, and I think the chain spec code section differs based on the profile. I would be curious to run the test again after you regenerate the dev_chain_spec.json, and also see if regenerating on my machine results in a different code section, or chain spec altogheter.

iulianbarbu · 2024-11-28T15:53:04Z

templates/minimal/node/tests/validate_dev_chain_spec.rs

@@ -0,0 +1,84 @@
+use codec::Decode;


I think having this file under /minimal/runtime is possible too. You'll need to add a [dev-dependencies] section under minimal/runtime/Cargo.toml:

[dev-dependencies] serde = { features = ["derive"], workspace = true, default-features = true } sc-service = { workspace = true } sc-chain-spec = { workspace = true }

templates/minimal/node/tests/validate_dev_chain_spec.rs

iulianbarbu · 2024-11-28T15:57:10Z

templates/minimal/node/Cargo.toml

+codec = { features = ["derive"], workspace = true }
 docify = { workspace = true }
 clap = { features = ["derive"], workspace = true }
 futures = { features = ["thread-pool"], workspace = true }
 futures-timer = { workspace = true }
+serde = { features = ["derive"], workspace = true, default-features = true }


No need for these after moving the test to minimal/runtime.

Co-authored-by: Iulian Barbu <[email protected]>

EleisonC added 5 commits November 16, 2024 23:26

Add development chain spec config

3743b9f

Add path chain_spec_path

1e833b4

Add path chain_spec_path

3345130

Add development chain spec for parachain template

97a1cc2

Update chain spec path

80669c1

EleisonC and others added 5 commits November 18, 2024 21:56

Merge branch 'master' into add-dev-chain-spec

7a060c5

Merge branch 'add-dev-chain-spec' of https://github.com/EleisonC/polk…

91e0ba8

…adot-sdk into add-dev-chain-spec

Revert "Update chain spec path"

d97247d

This reverts commit 80669c1.

update chain_spec_path

301ab59

Merge branch 'master' into add-dev-chain-spec

541b524

iulianbarbu self-requested a review November 19, 2024 15:03

iulianbarbu added T17-Templates This PR/Issue is related to templates T11-documentation This PR/Issue is related to documentation. labels Nov 19, 2024

iulianbarbu assigned EleisonC Nov 19, 2024

iulianbarbu added the R0-silent Changes should not be mentioned in any release notes label Nov 19, 2024

iulianbarbu reviewed Nov 20, 2024

View reviewed changes

EleisonC and others added 2 commits November 21, 2024 05:40

Merge branch 'master' into add-dev-chain-spec

02e431e

remove chain spec path not required

0146534

EleisonC and others added 2 commits November 28, 2024 01:32

Add runtime test for minimal dev chain spec

14a32f1

Signed-off-by: EleisonC <[email protected]>

Merge branch 'master' into add-dev-chain-spec

2c318e3

iulianbarbu reviewed Nov 28, 2024

View reviewed changes

EleisonC and others added 2 commits November 29, 2024 02:02

Update templates/minimal/node/tests/validate_dev_chain_spec.rs

cd66bbe

Co-authored-by: Iulian Barbu <[email protected]>

Update templates/minimal/node/tests/validate_dev_chain_spec.rs

419a9d8

Co-authored-by: Iulian Barbu <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add development chain-spec file for minimal/parachain templates for Omni Node compatibility #6529

Add development chain-spec file for minimal/parachain templates for Omni Node compatibility #6529

EleisonC commented Nov 18, 2024

cla-bot-2021 bot commented Nov 18, 2024

iulianbarbu left a comment

iulianbarbu Nov 20, 2024

iulianbarbu Nov 20, 2024

EleisonC commented Nov 20, 2024

iulianbarbu commented Nov 20, 2024

EleisonC commented Nov 21, 2024

iulianbarbu commented Nov 21, 2024 •

edited

Loading

EleisonC commented Nov 25, 2024 •

edited

Loading

EleisonC commented Nov 26, 2024

iulianbarbu commented Nov 27, 2024

EleisonC commented Nov 27, 2024

iulianbarbu left a comment

iulianbarbu Nov 28, 2024 •

edited

Loading

iulianbarbu Nov 28, 2024

Add development chain-spec file for minimal/parachain templates for Omni Node compatibility #6529

Are you sure you want to change the base?

Add development chain-spec file for minimal/parachain templates for Omni Node compatibility #6529

Conversation

EleisonC commented Nov 18, 2024

Description

Integration

Review Notes

cla-bot-2021 bot commented Nov 18, 2024

iulianbarbu left a comment

Choose a reason for hiding this comment

iulianbarbu Nov 20, 2024

Choose a reason for hiding this comment

iulianbarbu Nov 20, 2024

Choose a reason for hiding this comment

EleisonC commented Nov 20, 2024

iulianbarbu commented Nov 20, 2024

EleisonC commented Nov 21, 2024

iulianbarbu commented Nov 21, 2024 • edited Loading

EleisonC commented Nov 25, 2024 • edited Loading

EleisonC commented Nov 26, 2024

iulianbarbu commented Nov 27, 2024

EleisonC commented Nov 27, 2024

iulianbarbu left a comment

Choose a reason for hiding this comment

iulianbarbu Nov 28, 2024 • edited Loading

Choose a reason for hiding this comment

iulianbarbu Nov 28, 2024

Choose a reason for hiding this comment

iulianbarbu commented Nov 21, 2024 •

edited

Loading

EleisonC commented Nov 25, 2024 •

edited

Loading

iulianbarbu Nov 28, 2024 •

edited

Loading