Is it possible to enable runtime-benchmarks in node-template

[mmyyrroonn]

We built our work based on node-template months ago. Now, we plan to enable benchmark for our runtime and calculate accurate weight for each extrinsics.

However, the doc for benchmark is quite insufficient. I followed substrate's code and added related code into our repo. However, I met several issues, such as #5587, panic at this line

substrate/primitives/state-machine/src/lib.rs

Line 333 in 60e3a69

self.overlay.exit_runtime()

or Error: Input("Error executing runtime benchmark: Other(\"Instantiation: Export ext_benchmarking_commit_db_version_1 not found\")"). The PR's commit history for benchmark is quite complex and not easy to follow.

Is there any doc introducing how to enable benchmark in runtime and cli? Is it possilbe to enable runtime-bencharks in node-template?

[kaichaosun]

Check this code to see if it helps.

[mmyyrroonn]

@kaichaosun Thank you very much! 😄 let me have a look and try.

[mmyyrroonn]

@kaichaosun Hi. Thanks for your helpful response. I followed your PR and did another try again. However, I still encouter the previous question. Do you have any idea about the following error message?

Thread 'main' panicked at 'Runtime is not able to call this function in the overlay; qed: NotInRuntime', /home/myron/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/sp-state-machine-0.8.0-rc4/src/lib.rs:333

The stack trace is

   5: sp_state_machine::StateMachine<B,H,N,Exec>::execute_aux
   6: sp_state_machine::StateMachine<B,H,N,Exec>::execute
   7: frame_benchmarking_cli::command::<impl frame_benchmarking_cli::BenchmarkCmd>::run
   8: crust::command::run
   9: crust::main

I carefully check the differences between the demo PR and my implementation. One major difference is that we're based on 2.0.0-rc4. The second is we have two HostFunctions in native_executor_instance. The third one is that frame_benchmarks_cli cannot derive Clone. But I checked the code

substrate/bin/node/cli/src/cli.rs

Line 34 in b06a383

#[derive(Debug, StructOpt)]

in the master branch. I think it should be fine. Other files are almost same since I did some copy-paste work and double checked them again.

Appreciate any hint or suggestion. 🙏

[shawntabrizi]

@myronfanqiu This is something that was resolved with this PR:

#6513

Unfortunately RC-4 does not have this PR. So you will need to either bump or rollback your substrate version to get this working.

[mmyyrroonn]

@shawntabrizi Thanks for this information. I have two more questions about this issue.

• Looks like RC-4 is the latest release version. Is there any roadmap or ETA about the next release version?
• I tried to use the latest commit to have a try first. However, I met another issue. I'm a fresh man for Rust language. 😄 Please forgive me if it's a stupid question.
  In this PR, frame-support switch the version of smallvec from 1.4.0 to 1.4.1. sc-authority-discovery depends on libp2p v0.21. libp2p sets the dependency to smallvec='1.0'. I did a local test. It looks like setting smallvec='1.0' would choose the version 1.4.0 of smallvec, which will cause conflict with frame-support. I checked the lock file of substrate and the libp2p indeed relys on smallvec 1.4.1. Is it an issue or is there a way to set correct version of smallvec?

[mmyyrroonn]

@kaichaosun Thank you very much! Your solution works on rc2. Let me try one more latest dev version.

[mmyyrroonn]

@shawntabrizi I double checked the commit history. It seems like that the #6513 is ahead of rc4 release PR #6515 and should be in rc4. Do I miss anything?

[mmyyrroonn]

This commit fd55c45 which was ten days ago works for me as well. Thanks for all your help!!! 😄. Appreciate any ETA for the new version.

[kaichaosun]

New release is blocked by a Rust issue mentioned here.
I will close this issue for now, let us know if you still need help.