Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bridges] Prune messages from confirmation tx body, not from the on_idle #4944

Merged

Conversation

bkontur
Copy link
Contributor

@bkontur bkontur commented Jul 4, 2024

Original PR with more context: paritytech/parity-bridges-common#2211
Relates to: paritytech/parity-bridges-common#2210

TODO

  • fresh weighs for pallet_bridge_messages
  • add try_state for pallet_bridge_messages which checks for unpruned messages - relates to the comment
  • prepare migration, that prunes leftovers, which would be pruned eventually from on_idle the comment can be done also by set_storage / kill_storage or with OnRuntimeUpgrade implementatino when do_try_state_for_outbound_lanes detects problem.

Open question

  • Do we really need oldest_unpruned_nonce afterwards?
    • after the runtime upgrade and when do_try_state_for_outbound_lanes pass, we won't need any migrations here
      • we won't even need do_try_state_for_outbound_lanes

@bkontur bkontur added T15-bridges This PR/Issue is related to bridges. R0-silent Changes should not be mentioned in any release notes labels Jul 4, 2024
@bkontur bkontur changed the base branch from master to bko-bridges-v2-backport-refactoring July 4, 2024 13:41
@bkontur bkontur force-pushed the bko-bridges-v2-backport-pruning branch from e6dea3a to b352ea0 Compare July 4, 2024 13:44
@bkontur
Copy link
Contributor Author

bkontur commented Jul 4, 2024

bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages

bkontur and others added 4 commits July 4, 2024 16:09
Original PR with more context: paritytech/parity-bridges-common#2211


Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages
@bkontur bkontur force-pushed the bko-bridges-v2-backport-pruning branch from fa0035c to 7aa55e2 Compare July 4, 2024 21:05
@bkontur
Copy link
Contributor Author

bkontur commented Jul 4, 2024

bot clean

@bkontur bkontur marked this pull request as ready for review July 4, 2024 21:12
@paritytech-review-bot paritytech-review-bot bot requested a review from a team July 4, 2024 21:13
@bkontur bkontur self-assigned this Jul 4, 2024
bridges/modules/messages/src/lib.rs Outdated Show resolved Hide resolved
bridges/modules/messages/src/lib.rs Outdated Show resolved Hide resolved
const MAX_MESSAGES_ITERATION: u64 = 16;
let start_nonce =
expected_last_prunned_nonce.checked_sub(MAX_MESSAGES_ITERATION).unwrap_or(0);
for current_nonce in start_nonce..=expected_last_prunned_nonce {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with try_state(). Why do we use MAX_MESSAGES_ITERATION and not iterate all the way ?

Otherwise replacing the loop wit ha for looks good.

Copy link
Contributor Author

@bkontur bkontur Jul 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with try_state(). Why do we use MAX_MESSAGES_ITERATION and not iterate all the way ?

Because, on every runtime upgrade it would be more and more waste of time, e.g.

  • first time it would run from e.g. message with nonce 1000 to 0
  • another runtime upgrade (when more messages happen) it would run e.g. 3000 to 0 (we previously checked interval 1000 to 0)
  • another runtime upgrade (when more messages happen) it would run e.g. 6000 to 0 (we previously checked interval 3000 to 0)
  • and so on

Yes, that's why I updated description that maybe oldest_unpruned_nonce and do_try_state_for_outbound_lanes are not really needed.

Slava wanted to create migration, that remove all messages < oldest_unpruned_nonce, but I think his concert was valid only if we have already more than one lane, because on_idle he picked different/random lanes to prune depending on ActiveOutboundLanes::count, which is for fellows 1 :). I think MBM migration is too much for this and also for standard migration, we would need to handle consumed weights and store last pruned block somewhere else, because oldest_unpruned_nonce is incremented on delivery now.

But as I am thinking about it again now, I think we should really remove attribute lane.oldest_unpruned_nonce, because we prune on delivery and this attribute does not make sense. So, the removal of this attribute could cause storage version change and migration, which would clean everything :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, MBM migration and removed oldest_unpruned_nonce, would do the job :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't use oldest_unpruned_nonce for relayer, anyway we can leave also without these migration, the worst can happen that we will have just a few messages in the storage, which we don't need. We add monitoring or do some sanity checks with seaching storage manually after upgrade, and when few messages left, we could remove them with set_prefix / kill_storage.

@acatangiu acatangiu merged commit 7c875be into bko-bridges-v2-backport-refactoring Jul 10, 2024
162 of 165 checks passed
@acatangiu acatangiu deleted the bko-bridges-v2-backport-pruning branch July 10, 2024 10:13
@bkontur bkontur restored the bko-bridges-v2-backport-pruning branch July 11, 2024 13:58
bkontur added a commit that referenced this pull request Jul 11, 2024
github-merge-queue bot pushed a commit that referenced this pull request Jul 12, 2024
…cations (#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
TomaszWaszczyk pushed a commit to TomaszWaszczyk/polkadot-sdk that referenced this pull request Jul 13, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
github-merge-queue bot pushed a commit that referenced this pull request Aug 23, 2024
…dle (#5006)

(Please, do not merge until SA, reverted and restored of
#4944)

Original PR with more context:
paritytech/parity-bridges-common#2211
Relates to:
paritytech/parity-bridges-common#2210

## TODO

- [x] fresh weighs for `pallet_bridge_messages`
- [x] add `try_state` for `pallet_bridge_messages` which checks for
unpruned messages - relates to the
[comment](paritytech/parity-bridges-common#2211 (comment))
- [x] ~prepare migration, that prunes leftovers, which would be pruned
eventually from `on_idle` the
[comment](paritytech/parity-bridges-common#2211 (comment)
can be done also by `set_storage` / `kill_storage` or with
`OnRuntimeUpgrade` implementatino when `do_try_state_for_outbound_lanes`
detects problem.

## Open question

- [ ] Do we really need `oldest_unpruned_nonce` afterwards?
- after the runtime upgrade and when `do_try_state_for_outbound_lanes`
pass, we won't need any migrations here
    - we won't even need `do_try_state_for_outbound_lanes`
- please check comments bellow:
#4944 (comment)

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
R0-silent Changes should not be mentioned in any release notes T15-bridges This PR/Issue is related to bridges.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants