CLI Tool Explore Command: Runtime APIs, Events, Colorized outputs, sc…

…ale-typegen integration (#1290)

* restructure cli commands

* config: Add `SkipCheckIfFeeless` signed extension (#1264)

* config: Add `SkipCheckIfFeeless` signed extension

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Add extra extension to the default params

Signed-off-by: Alexandru Vasile <[email protected]>

* examples: Adjust signed extension example

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Extend SkipCheckIfFeeless with inner signed extension

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Configure SkipCheck with inner signed extension params

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Implement Deafult for SkipCheckIfFeelessParams with Option

Signed-off-by: Alexandru Vasile <[email protected]>

* examples: Fix example with proper extension

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Extend <T as Config>::AssetId with EncodeAsType and Clone

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Add SkipCheck with AssetTx

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Encode as type from metadata the inner signed extensions

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust examples

Signed-off-by: Alexandru Vasile <[email protected]>

* blocks: Use `SkipCheckIfFeeless` for decoding the tip of extensions

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Decode `SkipCheckIfFeeless` with `Self`

Signed-off-by: Alexandru Vasile <[email protected]>

* tests: Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Descriptive errors for building `SkipCheckIfFeeless`

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Add docs for extra error types

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Add extra derives to signed extensions

Signed-off-by: Alexandru Vasile <[email protected]>

* config: Use `Default::default` to simplify type init

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Replace removed lint (#1270)

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Add support for multi-chain usecase (#1238)

* lightclient: Make `smoldot::chainID` part of the RPC requests

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Make `BackgroundTask` generic over `PlatformRef` and chain

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Construct from raw smoldot and target different chains

Signed-off-by: Alexandru Vasile <[email protected]>

* testing: Update cargo lock for wasm tests

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Reuse `new_from_client` method and removed unused imports

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Reexport smoldot client and RPC objects used in pub
interface

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Adjust `new_from_client` interface

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Extend background to poll over multiple RPC objects

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Build light client from raw and target different chains

Signed-off-by: Alexandru Vasile <[email protected]>

* artifacts: Add demo chain specs

Signed-off-by: Alexandru Vasile <[email protected]>

* artifacts: Move artifacts to dedicated folder

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Use SelectAll to drive all streams

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Fetch initial data from the target chain

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Reexport other smoldot objects

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Target chain with potentially different config

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt/rpc: Log chainID for debugging

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt/examples: Add smoldot client with parachain example

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Propagate chain ID together with rpc responses object

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Multiplex responses by request ID and chain ID

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Add raw light client builder

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Add cargo feature flag for parachains example

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Derive default for internal structure

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Guard reexports by std feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* Update subxt/src/client/light_client/mod.rs

Co-authored-by: James Wilson <[email protected]>

* lightclient: Update the builder pattern and chain targetting

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Fix documentation

Signed-off-by: Alexandru Vasile <[email protected]>

* Provide more insightful docs wrt native/wasm panics

Signed-off-by: Alexandru Vasile <[email protected]>

* examples: Adjust comment location

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Refactor UniqueChainId into the background task

Signed-off-by: Alexandru Vasile <[email protected]>

* Update lightclient/src/background.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Update subxt/src/client/light_client/builder.rs

Co-authored-by: James Wilson <[email protected]>

* lightclient: Update docs wrt panics

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Update docs wrt to smoldot instance -> client

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Use IntoIter instead of Iterator

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Adjsut docs wrt [`Self::new_from_client`]

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Remove RawRpc from LightClient in favor of chainID

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Reexport everything under smoldot module

Signed-off-by: Alexandru Vasile <[email protected]>

* artifacts: Use stateRootHash instead of genesis.raw

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>

* Bump futures from 0.3.28 to 0.3.29 (#1272)

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.28 to 0.3.29.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](rust-lang/futures-rs@0.3.28...0.3.29)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump zeroize from 1.6.0 to 1.7.0 (#1274)

Bumps [zeroize](https://github.com/RustCrypto/utils) from 1.6.0 to 1.7.0.
- [Commits](https://github.com/RustCrypto/utils/commits)

---
updated-dependencies:
- dependency-name: zeroize
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump tracing-subscriber from 0.3.17 to 0.3.18 (#1275)

Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](tokio-rs/tracing@tracing-subscriber-0.3.17...tracing-subscriber-0.3.18)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump tracing-subscriber from 0.3.17 to 0.3.18 (#1275)

Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](tokio-rs/tracing@tracing-subscriber-0.3.17...tracing-subscriber-0.3.18)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump getrandom from 0.2.10 to 0.2.11 (#1273)

Bumps [getrandom](https://github.com/rust-random/getrandom) from 0.2.10 to 0.2.11.
- [Changelog](https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md)
- [Commits](rust-random/getrandom@v0.2.10...v0.2.11)

---
updated-dependencies:
- dependency-name: getrandom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* impl RpcClientT for Arc<T> and Box<T> (#1277)

* impl RpcClientT for Arc<WsClient>

* fix grumbles: impl for Box<T> and Arc<T>

* grumbles: move RpcClientT impls

* first iteration of using scale_typegen

* introduce indoc for formatting

* calls, constants and home are cleaner now

* added event subcommand

* show runtime apis working

* add better code formatting

* fix style

* adjust tests, use owo_colorize to not add extra dependency

* fmt

* adjust docs

* move scale-typegen-description dependency to workspace

* improve `substrate-compat` (#1265)

* improve `substrate-compat`

* From => Into

---------

Co-authored-by: James Wilson <[email protected]>

* Bump proc-macro2 from 1.0.69 to 1.0.70 (#1292)

Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](dtolnay/proc-macro2@1.0.69...1.0.70)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump serde from 1.0.192 to 1.0.193 (#1291)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.192 to 1.0.193.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](serde-rs/serde@v1.0.192...v1.0.193)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* lightclient: Fix wasm socket closure called after being dropped (#1289)

* lightclient: Close wasm socket while dropping from connecting state

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Construct one time only closures

Signed-off-by: Alexandru Vasile <[email protected]>

* testing: Enable console logs for lightclient WASM testing

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Separate wakes and check connectivity on poll_read

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Close the socket depending on internal state

Signed-off-by: Alexandru Vasile <[email protected]>

* Revert "lightclient: Separate wakes and check connectivity on poll_read"

This reverts commit 8660940.

* lightclient: Return pending if socket is opening from poll_read

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Close the socket on `poll_close`

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Reset closures on Drop to avoid recursive invokation

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Close the socket if not already closing

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>

* workflows: Install rustup component for building substrate (#1295)

Signed-off-by: Alexandru Vasile <[email protected]>

* cli: Command to fetch chainSpec and optimise its size (#1278)

* cli: Add chainSpec command

Signed-off-by: Alexandru Vasile <[email protected]>

* cli/chainSpec: Move to dedicated module

Signed-off-by: Alexandru Vasile <[email protected]>

* cli: Compute the state root hash

Signed-off-by: Alexandru Vasile <[email protected]>

* cli: Remove code substitutes

Signed-off-by: Alexandru Vasile <[email protected]>

* artifacts: Update polkadot.json

Signed-off-by: Alexandru Vasile <[email protected]>

* scripts: Generate the chain spec

Signed-off-by: Alexandru Vasile <[email protected]>

* cli: Remove testing artifacts

Signed-off-by: Alexandru Vasile <[email protected]>

* cli: Fix clippy

Signed-off-by: Alexandru Vasile <[email protected]>

* cli: Apply rustfmt

Signed-off-by: Alexandru Vasile <[email protected]>

* cli: Introduce feature flag for smoldot dependency

Signed-off-by: Alexandru Vasile <[email protected]>

* cli: Rename chain-spec to chain-spec-pruning

Signed-off-by: Alexandru Vasile <[email protected]>

* scripts: Update chain-spec command

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>

* update to new scale-typegen interfaces

* use released version of scale-typegen

* Merge branch 'master' into tadeohepperle/cli-support-runtime-apis

* remove unused debug file

* resolve merge errors

* adjustments

* constants file adjustment

* method renaming

* fix issue with encoding runtime api params

* Add logging to submit_transaction and unstable driver, and ensure unpin evs complete

* panic if None returned from subscription too, also with stats

* change panic to Err just to be on the safe side

* clippy

* make long tests run only after clippy + fmt pass

* megre in light client test change pr

* chore(subxt/src): typo fix (#1370)

* rpcmethods

* followstr

* mod and else

* Weekly Cronjob fetching artifacts and generating polkadot.rs file. (#1352)

* github CI action cronjob

* add commit message

* fix the CI yml files

* binary crate for CI script with substrate-runner

* update the CI script

* correct the artifacts script

* remove bash script

* lightclient(fix): Ensure lightclient chainSpec is at least one block old (#1372)

* testing(fix): Ensure lightclient chainSpec is at least one block old

Signed-off-by: Alexandru Vasile <[email protected]>

* Revert "testing(fix): Ensure lightclient chainSpec is at least one block old"

This reverts commit 0eafcb2.

* lightclient(fix): Ensure lightclient chainSpec is at least one block old

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Link smoldot issue

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Use tokio under lightclient feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Do not sleep on errors to fetch the chainSpec

Signed-off-by: Alexandru Vasile <[email protected]>

* artifacts: Remove test file

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Subscribe to two finalized blocks

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Revert cargo toml

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>

* ci: Reduce the light client timeout to 15 minutes (#1373)

* ci: Reduce the light client timpeut to 15 seconds

Signed-off-by: Alexandru Vasile <[email protected]>

* ci: Use ubuntu-latest for light-client tests

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>

* actually only wait for machete+fmt, clippy can be much slower

* update CI file from Alex PR

* resolve clippy err

* Try a few RPC nodes in case one of them is not working

* fix submit_transaction debug logging of message

* Improve Signed Extension and Block Decoding Examples/Book (#1357)

* asset hub example and book adjustment

* formatting

* recursive derives

* polkadot monitor example and book adjustments

* formatting

* adjust docs and examples, add dynamic example

* james suggestions

* fmt

* chore(subxt/src): typo fix (#1370)

* rpcmethods

* followstr

* mod and else

* Weekly Cronjob fetching artifacts and generating polkadot.rs file. (#1352)

* github CI action cronjob

* add commit message

* fix the CI yml files

* binary crate for CI script with substrate-runner

* update the CI script

* correct the artifacts script

* remove bash script

---------

Co-authored-by: James Wilson <[email protected]>
Co-authored-by: Pan chao <[email protected]>

* fix formatting of returned sections

* make storage use execute flag as well

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: yjh <[email protected]>
Co-authored-by: Pan chao <[email protected]>

Cargo.toml

@@ -70,6 +70,7 @@ getrandom = { version = "0.2", default-features = false }
 hex = "0.4.3"
 heck = "0.4.1"
 impl-serde = { version = "0.4.0" }
+indoc = "2"
 jsonrpsee = { version = "0.21" }
 pretty_assertions = "1.4.0"
 primitive-types = { version = "0.12.2", default-features = false, features = ["codec", "scale-info", "serde"] }
@@ -97,6 +98,7 @@ wasm-bindgen-test = "0.3.24"
 which = "5.0.0"
 scale-typegen-description = "0.1.0"
 scale-typegen = "0.1.1"
+strip-ansi-escapes = "0.2.0"
 
 # Light client support:
 smoldot = { version = "0.16.0", default-features = false }

cli/Cargo.toml

@@ -40,5 +40,12 @@ scale-value = { workspace = true }
 syn = { workspace = true }
 jsonrpsee = { workspace = true, features = ["async-client", "client-ws-transport-native-tls", "http-client"] }
 tokio = { workspace = true, features = ["rt-multi-thread"] }
+scale-typegen-description = { workspace = true }
+heck = { workspace = true }
+indoc = { workspace = true }
 thiserror = { workspace = true }
-smoldot = { workspace = true, optional = true}
+smoldot = { workspace = true, optional = true }
+
+[dev-dependencies]
+strip-ansi-escapes = { workspace = true }
+pretty_assertions = { workspace = true }