Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bridges - changes for Bridges V2 - compact proofs #4729

Draft
wants to merge 77 commits into
base: master
Choose a base branch
from
Draft

Bridges - changes for Bridges V2 - compact proofs #4729

wants to merge 77 commits into from

Conversation

bkontur
Copy link
Contributor

@bkontur bkontur commented Jun 7, 2024 •
edited
Loading

This PR contains migrated code from the Bridges V2 branch from old parity-bridges-common repo.

Summary

TBD

Relates to: paritytech/parity-bridges-common#3020

TODO

  • add prdoc
  • generate weights
  • check/add migration if needed (at least transaction_version should change)
  • ensure that we can run relayer in setup like: Kusama with compact proofs and Polkadot without compact proofs - probably custom generate_submit_finality_proof_call_builder ?
  • comparision performance? lower fees?

Weights comparision

BridgeHubRoocco

(--threshold 0.5)

subweight compare commits \
         --path-pattern "./cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/**/weights/**/*.rs" \
         --format markdown --no-color  \
         --change added changed removed  \
        --method asymptotic --ignore-errors  \
        --strip-path-prefix="cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/src/weights/" \
        --threshold 0.5 \
         remotes/origin/master \
         origin/bko-bridges-v2-compact-proofs

TODO:

BridgeHubWestend

(--threshold 0.5)

subweight compare commits \
         --path-pattern "./cumulus/parachains/runtimes/bridge-hubs/bridge-hub-westend/**/weights/**/*.rs" \
         --format markdown --no-color  \
         --change added changed removed  \
        --method asymptotic --ignore-errors  \
        --strip-path-prefix="cumulus/parachains/runtimes/bridge-hubs/bridge-hub-westend/src/weights/" \
        --threshold 0.5 \
         remotes/origin/master \
         origin/bko-bridges-v2-compact-proofs

TODO:

@bkontur bkontur added the T15-bridges This PR/Issue is related to bridges. label Jun 7, 2024
@bkontur bkontur force-pushed the bko-bridges-v2-compact-proofs branch 2 times, most recently from 46f3c36 to 79e3a79 Compare June 13, 2024 10:23
bkontur
bkontur commented Jun 13, 2024
View reviewed changes
bridges/primitives/messages/src/target_chain.rs
///
/// - hash of finalized header;
///
/// - storage proof of messages and (optionally) outbound lane state;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@serban300 what is "(optionally) outbound lane state" here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's the outbound_lane_data. I'm not completely sure.

@bkontur bkontur force-pushed the bko-bridges-v2-compact-proofs branch 2 times, most recently from f0f50af to f4008e5 Compare June 14, 2024 11:28
@bkontur
Copy link
Contributor Author

bkontur commented Jun 14, 2024

bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_parachains
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_relayers
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_grandpa
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_parachains
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_relayers

Base automatically changed from bko-bridges-backports-and-nits to master June 14, 2024 11:55
@bkontur bkontur force-pushed the bko-bridges-v2-compact-proofs branch from f4008e5 to 39fcf70 Compare June 14, 2024 13:26
@bkontur
Copy link
Contributor Author

bkontur commented Jun 14, 2024

bot clean

@bkontur
Copy link
Contributor Author

bkontur commented Jun 14, 2024

bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_parachains
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_relayers
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_grandpa
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_parachains
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_relayers

@bkontur
Copy link
Contributor Author

bkontur commented Jun 14, 2024

bot clean

@bkontur bkontur force-pushed the bko-bridges-v2-compact-proofs branch from 39fcf70 to 0e194bf Compare June 14, 2024 20:22
@bkontur
Copy link
Contributor Author

bkontur commented Jun 14, 2024

bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_parachains
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_relayers
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_grandpa
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_parachains
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_relayers

@bkontur bkontur force-pushed the bko-bridges-v2-compact-proofs branch from 0e194bf to 464d212 Compare June 20, 2024 08:24
@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot clean
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa

@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot clean
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa

@bkontur bkontur force-pushed the bko-bridges-v2-compact-proofs branch from 464d212 to 55b3aea Compare June 20, 2024 09:23
@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot clean
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa

3 similar comments
@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot clean
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa

@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot clean
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa

@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot clean
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa

@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot clean
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa

@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot clean
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa

@bkontur
Copy link
Contributor Author

bkontur commented Jun 20, 2024

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_parachains
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_relayers
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_grandpa
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_parachains
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_relayers

@bkontur bkontur mentioned this pull request Jun 21, 2024
Draft
8 tasks
serban300 and others added 3 commits June 26, 2024 21:08
@serban300 @bkontur
Use compact proofs for messages delivery (#2155)
bc0f814 
* Define Chain::STATE_VERSION

* Add vec_db module

* Use VecDb instead of StorageProof for message delivery

* Make sure that the TrustedVecDb is sorted

* Address review comments

* Run benchmarks on parent commit

* Run benchmarks with new code

* Fix test

* Fix code review comments
@svyatonik @bkontur
Moved FromBridgedChainMessagesDeliveryProof to bp-messages (#2169)
18b5b22 
* moved FromBridgedChainMessagesDeliveryProof to bp-messages

* spelling

* fix benchmarks compilation
@svyatonik @bkontur
Moved FromBridgedChainMessagesProof to bp-messages (#2170)
379a600 
* moved FromBridgedChainMessagesProof to bp-messages

* fmt
command-bot added 2 commits July 5, 2024 15:03
Merge branch 'bko-bridges-v2-backport-refactoring' of https://github.…
4e3d4cb 
…com/paritytech/polkadot-sdk into HEAD
".git/.scripts/commands/bench/bench.sh" --subcommand=xcm --runtime=br…
3df9702 
…idge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_xcm_benchmarks::generic
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=xcm --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_xcm_benchmarks::generic has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640478 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640478/artifacts/download.

command-bot added 2 commits July 5, 2024 16:20
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
e72b903 
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
12dd903 
…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640472 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640472/artifacts/download.

@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640474 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640474/artifacts/download.

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
80ee268 
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640469 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640469/artifacts/download.

command-bot added 2 commits July 5, 2024 16:21
".git/.scripts/commands/bench/bench.sh" --subcommand=xcm --runtime=br…
8c999d2 
…idge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_xcm_benchmarks::generic
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
221a322 
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640470 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640470/artifacts/download.

@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=xcm --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_xcm_benchmarks::generic has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640477 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640477/artifacts/download.

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
6145337 
…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640476 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640476/artifacts/download.

command-bot added 2 commits July 5, 2024 16:22
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
b737584 
…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
423526f 
…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640475 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640475/artifacts/download.

@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640473 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640473/artifacts/download.

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
4ac91fe 
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640471 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640471/artifacts/download.

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
41769a6 
…=asset-hub-rococo --runtime_dir=assets --target_dir=cumulus --pallet=pallet_xcm_bridge_hub_router
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=asset-hub-rococo --runtime_dir=assets --target_dir=cumulus --pallet=pallet_xcm_bridge_hub_router has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640479 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640479/artifacts/download.

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
72fbc87 
…=asset-hub-westend --runtime_dir=assets --target_dir=cumulus --pallet=pallet_xcm_bridge_hub_router
@command-bot
Copy link

command-bot bot commented Jul 5, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=asset-hub-westend --runtime_dir=assets --target_dir=cumulus --pallet=pallet_xcm_bridge_hub_router has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640480 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6640480/artifacts/download.

@bkontur
Change prove_storage to return complete proof (proof, keys, state_r…
97a5146 
…oot)
@bkontur
Make relayer proofs more generic - new stub layer for proofs that are…
9b358d1 
… convertible to different kind of proofs (for backwards compatibility, that we need for relayers, when chains are upgraded asynchrounously)
@paritytech-cicd-pr
Copy link

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: cargo-clippy
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6647428

@bkontur
Make relayer proofs more generic - new stub layer for proofs that are…
640290e 
… convertible to different kind of proofs (for backwards compatibility, that we need for relayers, when chains are upgraded asynchrounously) - part 2
Base automatically changed from bko-bridges-v2-backport-refactoring to master July 12, 2024 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@serban300 serban300 serban300 left review comments

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
T15-bridges This PR/Issue is related to bridges.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@bkontur @serban300 @paritytech-cicd-pr @svyatonik