Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runtime Upgrade ref docs and Single Block Migration example pallet #1554

Merged
merged 110 commits into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
a9673c7
improve metadata docs
liamaharon Sep 13, 2023
e22a2e2
improve hooks docs
liamaharon Sep 13, 2023
c6a6984
add storage_alias docs
liamaharon Sep 13, 2023
b617ce9
wip: storage migrations example pallet
liamaharon Sep 13, 2023
37e45d4
dont export unversioned migrations
liamaharon Sep 13, 2023
22fe899
patterns preventing accidental usage of unversioned migrations
liamaharon Sep 14, 2023
3ef233f
Merge branch 'master' of github.com:paritytech/polkadot-sdk into liam…
liamaharon Sep 14, 2023
76c9150
private unversioned migrations
liamaharon Sep 14, 2023
d98a0fd
docs
liamaharon Sep 14, 2023
48100f0
docs
liamaharon Sep 14, 2023
986f0bd
note about pub(crate)
liamaharon Sep 14, 2023
548316c
migration safely docs
liamaharon Sep 14, 2023
4925243
typo
liamaharon Sep 14, 2023
7d75a75
docs
liamaharon Sep 14, 2023
6858f3c
docs
liamaharon Sep 14, 2023
8446ad2
doctest false
liamaharon Sep 14, 2023
bdb1ae5
note on chopsticks
liamaharon Sep 14, 2023
35b08fd
fix society tests
liamaharon Sep 14, 2023
1fa5207
zepter
liamaharon Sep 14, 2023
ebf4cd5
fix feature propagation
liamaharon Sep 14, 2023
3f55db7
add licences
liamaharon Sep 14, 2023
ca0e4e3
format features
liamaharon Sep 14, 2023
34da4a0
fix doc visibility issue
liamaharon Sep 14, 2023
009a958
improve docs
liamaharon Sep 14, 2023
1aa3555
fix std propagataion
liamaharon Sep 14, 2023
fc08b0e
zepter
liamaharon Sep 14, 2023
01b780a
fix broken link
liamaharon Sep 14, 2023
adcfeaf
dont ref private item
liamaharon Sep 14, 2023
655b376
dont ref private doc item
liamaharon Sep 14, 2023
8335171
doc build
liamaharon Sep 14, 2023
3fff7fb
Update substrate/frame/examples/storage-migrations/src/mock.rs
liamaharon Sep 16, 2023
3bd0ef2
Update substrate/frame/examples/src/lib.rs
liamaharon Sep 17, 2023
1a54755
docify
liamaharon Sep 17, 2023
222ed3a
derive_impl
liamaharon Sep 17, 2023
4a070fd
fix derive_impl
liamaharon Sep 20, 2023
f66f9e2
improve docs
liamaharon Sep 20, 2023
70a9da4
set dbweight
liamaharon Sep 20, 2023
d915026
clean up pallet
liamaharon Sep 20, 2023
a596074
module visibility
liamaharon Sep 20, 2023
0cf19ac
fix comment
liamaharon Sep 20, 2023
fe92288
fix comment
liamaharon Sep 20, 2023
0dc8a60
fix comment
liamaharon Sep 20, 2023
e3e4940
better naming
liamaharon Sep 20, 2023
0fe21d5
better naming
liamaharon Sep 20, 2023
92e43ac
fix wasm build
liamaharon Sep 20, 2023
55f876d
cargo fmt
liamaharon Sep 20, 2023
4819209
format features
liamaharon Sep 20, 2023
59d7352
set uniques storage version
liamaharon Sep 20, 2023
f16f628
fix asset hub storage versions
liamaharon Sep 20, 2023
bbae7ef
Merge branch 'master' into liam-migrations-reference-docs
liamaharon Nov 28, 2023
7d4da68
merge
liamaharon Nov 28, 2023
eaa6aca
update cargo.lock
liamaharon Nov 28, 2023
8cafd48
reset polkadot-asset-hub-runtime
liamaharon Nov 28, 2023
a6c6781
fix merge
liamaharon Nov 28, 2023
fbdb77c
fix up merge
liamaharon Nov 28, 2023
e9aa91a
remove kusama runtime
liamaharon Nov 28, 2023
d135244
Merge branch 'master' into liam-migrations-reference-docs
liamaharon Nov 29, 2023
7a23c5f
update cargo.lock
liamaharon Nov 29, 2023
c120f47
fix build
liamaharon Nov 29, 2023
a4c2ca3
rename single block migrations
liamaharon Nov 29, 2023
4275503
runtime upgrade ref docs
liamaharon Nov 30, 2023
48605c7
Merge branch 'master' into liam-migrations-reference-docs
liamaharon Nov 30, 2023
2c014e7
version
liamaharon Nov 30, 2023
9f46c00
comment
liamaharon Nov 30, 2023
ef12e3e
note on version
liamaharon Nov 30, 2023
4881a0a
berevity
liamaharon Nov 30, 2023
e5ff5b2
brevity
liamaharon Nov 30, 2023
3f31c6b
Update substrate/frame/examples/single-block-migrations/Cargo.toml
liamaharon Dec 1, 2023
802aae3
Update substrate/frame/examples/single-block-migrations/src/lib.rs
liamaharon Dec 1, 2023
7aca6b6
Update substrate/frame/support/procedural/src/lib.rs
liamaharon Dec 1, 2023
564664e
Update substrate/frame/examples/single-block-migrations/src/runtime_u…
liamaharon Dec 1, 2023
dfa7f36
Update substrate/frame/examples/single-block-migrations/src/lib.rs
liamaharon Dec 1, 2023
b6d71a8
Update substrate/frame/examples/single-block-migrations/src/migration…
liamaharon Dec 1, 2023
4651a7e
Merge branch 'master' into liam-migrations-reference-docs
liamaharon Dec 1, 2023
eaa427a
format toml
liamaharon Dec 1, 2023
406fad7
address comment
liamaharon Dec 1, 2023
4be8f4d
address comment
liamaharon Dec 1, 2023
a8df9cd
address comment
liamaharon Dec 1, 2023
8f32ece
add benchmark note
liamaharon Dec 1, 2023
ec3ce8c
take
liamaharon Dec 1, 2023
0ae45d8
fix typo
liamaharon Dec 1, 2023
f5ac289
address comment
liamaharon Dec 1, 2023
7f21f42
current storage version -> in_code storage version
liamaharon Dec 1, 2023
050f465
try fix ci
liamaharon Dec 1, 2023
91f80eb
rename
liamaharon Dec 1, 2023
66d2745
feature propagarion
liamaharon Dec 1, 2023
1a2d834
fix unused import wanrings
liamaharon Dec 1, 2023
69e4209
fix typo
liamaharon Dec 1, 2023
6c50902
Merge branch 'master' of github.com:paritytech/polkadot-sdk into liam…
liamaharon Dec 7, 2023
52963fa
fix merge
liamaharon Dec 7, 2023
dfe32c5
move reference doc to correct place
liamaharon Dec 7, 2023
2d08759
".git/.scripts/commands/fmt/fmt.sh"
Dec 7, 2023
2fadaad
prdoc
liamaharon Dec 7, 2023
fc9d605
Merge branch 'liam-migrations-reference-docs' of github.com:paritytec…
liamaharon Dec 7, 2023
8108b8e
fix prdoc
liamaharon Dec 7, 2023
a655cd8
Merge branch 'master' into liam-migrations-reference-docs
liamaharon Dec 7, 2023
03b7735
keep current_storage_version around for 3 months
liamaharon Dec 7, 2023
27414b4
Merge branch 'liam-migrations-reference-docs' of github.com:paritytec…
liamaharon Dec 7, 2023
a6c1dc6
improve deprecation msg
liamaharon Dec 7, 2023
d8f5a38
Merge branch 'master' into liam-migrations-reference-docs
liamaharon Jan 18, 2024
319b32d
update cargo.lock
liamaharon Jan 18, 2024
85860cf
Merge branch 'master' into liam-migrations-reference-docs
liamaharon Feb 28, 2024
65082f9
update cargo.lock
liamaharon Feb 28, 2024
5b0bc65
rename
liamaharon Feb 28, 2024
d76e813
address comments
liamaharon Feb 28, 2024
26bf39f
address comments
liamaharon Feb 28, 2024
c588805
address comments and make compile
liamaharon Feb 28, 2024
3aa6ccd
Update docs/sdk/src/reference_docs/frame_runtime_upgrades_and_migrati…
liamaharon Feb 28, 2024
0f63e03
address comment
liamaharon Feb 28, 2024
a97071b
Merge branch 'liam-migrations-reference-docs' of github.com:paritytec…
liamaharon Feb 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,7 @@ members = [
"substrate/frame/examples/kitchensink",
"substrate/frame/examples/offchain-worker",
"substrate/frame/examples/split",
"substrate/frame/examples/storage-migrations",
"substrate/frame/executive",
"substrate/frame/fast-unstake",
"substrate/frame/glutton",
Expand Down
89 changes: 50 additions & 39 deletions polkadot/runtime/common/src/assigned_slots/migration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,51 +23,62 @@ use frame_support::ensure;
use sp_std::vec::Vec;

pub mod v1 {

use super::*;
pub struct MigrateToV1<T>(sp_std::marker::PhantomData<T>);
impl<T: Config> OnRuntimeUpgrade for MigrateToV1<T> {
#[cfg(feature = "try-runtime")]
fn pre_upgrade() -> Result<Vec<u8>, sp_runtime::TryRuntimeError> {
let onchain_version = Pallet::<T>::on_chain_storage_version();
ensure!(onchain_version < 1, "assigned_slots::MigrateToV1 migration can be deleted");
Ok(Default::default())
}

fn on_runtime_upgrade() -> frame_support::weights::Weight {
let onchain_version = Pallet::<T>::on_chain_storage_version();
if onchain_version < 1 {
const MAX_PERMANENT_SLOTS: u32 = 100;
const MAX_TEMPORARY_SLOTS: u32 = 100;
mod version_unchecked {
use super::*;

<MaxPermanentSlots<T>>::put(MAX_PERMANENT_SLOTS);
<MaxTemporarySlots<T>>::put(MAX_TEMPORARY_SLOTS);
// Return the weight consumed by the migration.
T::DbWeight::get().reads_writes(1, 3)
} else {
log::info!(target: LOG_TARGET, "MigrateToV1 should be removed");
T::DbWeight::get().reads(1)
pub struct MigrateV0ToV1<T>(sp_std::marker::PhantomData<T>);
impl<T: Config> OnRuntimeUpgrade for MigrateV0ToV1<T> {
#[cfg(feature = "try-runtime")]
fn pre_upgrade() -> Result<Vec<u8>, sp_runtime::TryRuntimeError> {
let onchain_version = Pallet::<T>::on_chain_storage_version();
ensure!(
onchain_version < 1,
"assigned_slots::MigrateToV1 migration can be deleted"
);
Ok(Default::default())
}
}

#[cfg(feature = "try-runtime")]
fn post_upgrade(_state: Vec<u8>) -> Result<(), sp_runtime::TryRuntimeError> {
let onchain_version = Pallet::<T>::on_chain_storage_version();
ensure!(onchain_version == 1, "assigned_slots::MigrateToV1 needs to be run");
assert_eq!(<MaxPermanentSlots<T>>::get(), 100);
assert_eq!(<MaxTemporarySlots<T>>::get(), 100);
Ok(())
fn on_runtime_upgrade() -> frame_support::weights::Weight {
let onchain_version = Pallet::<T>::on_chain_storage_version();
if onchain_version < 1 {
const MAX_PERMANENT_SLOTS: u32 = 100;
const MAX_TEMPORARY_SLOTS: u32 = 100;

<MaxPermanentSlots<T>>::put(MAX_PERMANENT_SLOTS);
<MaxTemporarySlots<T>>::put(MAX_TEMPORARY_SLOTS);
// Return the weight consumed by the migration.
T::DbWeight::get().reads_writes(1, 3)
} else {
log::info!(target: LOG_TARGET, "MigrateToV1 should be removed");
T::DbWeight::get().reads(1)
}
}

#[cfg(feature = "try-runtime")]
fn post_upgrade(_state: Vec<u8>) -> Result<(), sp_runtime::TryRuntimeError> {
let onchain_version = Pallet::<T>::on_chain_storage_version();
ensure!(onchain_version == 1, "assigned_slots::MigrateToV1 needs to be run");
assert_eq!(<MaxPermanentSlots<T>>::get(), 100);
assert_eq!(<MaxTemporarySlots<T>>::get(), 100);
Ok(())
}
}
}

/// [`MigrateToV1`] wrapped in a
/// [`VersionedMigration`](frame_support::migrations::VersionedMigration), ensuring the
/// migration is only performed when on-chain version is 0.
pub type VersionCheckedMigrateToV1<T> = frame_support::migrations::VersionedMigration<
0,
1,
MigrateToV1<T>,
Pallet<T>,
<T as frame_system::Config>::DbWeight,
>;
pub mod versioned {
use super::*;

/// `version_unchecked::MigrateV0ToV1` wrapped in a
/// [`VersionedMigration`](frame_support::migrations::VersionedMigration), ensuring the
/// migration is only performed when on-chain version is 0.
pub type MigrateV0ToV1<T> = frame_support::migrations::VersionedMigration<
0,
1,
version_unchecked::MigrateV0ToV1<T>,
Pallet<T>,
<T as frame_system::Config>::DbWeight,
>;
}
}
84 changes: 46 additions & 38 deletions polkadot/runtime/common/src/paras_registrar/migration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,47 +24,55 @@ pub struct ParaInfoV1<Account, Balance> {
locked: bool,
}

pub struct VersionUncheckedMigrateToV1<T, UnlockParaIds>(
sp_std::marker::PhantomData<(T, UnlockParaIds)>,
);
impl<T: Config, UnlockParaIds: Contains<ParaId>> OnRuntimeUpgrade
for VersionUncheckedMigrateToV1<T, UnlockParaIds>
{
fn on_runtime_upgrade() -> Weight {
let mut count = 0u64;
Paras::<T>::translate::<ParaInfoV1<T::AccountId, BalanceOf<T>>, _>(|key, v1| {
count.saturating_inc();
Some(ParaInfo {
manager: v1.manager,
deposit: v1.deposit,
locked: if UnlockParaIds::contains(&key) { None } else { Some(v1.locked) },
})
});
pub mod v1 {
use super::*;

log::info!(target: "runtime::registrar", "Upgraded {} storages to version 1", count);
T::DbWeight::get().reads_writes(count, count)
}
mod version_unchecked {
use super::*;

#[cfg(feature = "try-runtime")]
fn pre_upgrade() -> Result<Vec<u8>, sp_runtime::TryRuntimeError> {
Ok((Paras::<T>::iter_keys().count() as u32).encode())
}
pub struct MigrateV0ToV1<T, UnlockParaIds>(sp_std::marker::PhantomData<(T, UnlockParaIds)>);
impl<T: Config, UnlockParaIds: Contains<ParaId>> OnRuntimeUpgrade
for MigrateV0ToV1<T, UnlockParaIds>
{
fn on_runtime_upgrade() -> Weight {
let mut count = 0u64;
Paras::<T>::translate::<ParaInfoV1<T::AccountId, BalanceOf<T>>, _>(|key, v1| {
count.saturating_inc();
Some(ParaInfo {
manager: v1.manager,
deposit: v1.deposit,
locked: if UnlockParaIds::contains(&key) { None } else { Some(v1.locked) },
})
});

log::info!(target: "runtime::registrar", "Upgraded {} storages to version 1", count);
T::DbWeight::get().reads_writes(count, count)
}

#[cfg(feature = "try-runtime")]
fn pre_upgrade() -> Result<Vec<u8>, sp_runtime::TryRuntimeError> {
Ok((Paras::<T>::iter_keys().count() as u32).encode())
}

#[cfg(feature = "try-runtime")]
fn post_upgrade(state: Vec<u8>) -> Result<(), sp_runtime::TryRuntimeError> {
let old_count = u32::decode(&mut &state[..]).expect("Known good");
let new_count = Paras::<T>::iter_values().count() as u32;
#[cfg(feature = "try-runtime")]
fn post_upgrade(state: Vec<u8>) -> Result<(), sp_runtime::TryRuntimeError> {
let old_count = u32::decode(&mut &state[..]).expect("Known good");
let new_count = Paras::<T>::iter_values().count() as u32;

ensure!(old_count == new_count, "Paras count should not change");
Ok(())
ensure!(old_count == new_count, "Paras count should not change");
Ok(())
}
}
}
}

pub type VersionCheckedMigrateToV1<T, UnlockParaIds> =
frame_support::migrations::VersionedMigration<
0,
1,
VersionUncheckedMigrateToV1<T, UnlockParaIds>,
super::Pallet<T>,
<T as frame_system::Config>::DbWeight,
>;
pub mod versioned {
use super::*;
pub type MigrateV0ToV1<T, UnlockParaIds> = frame_support::migrations::VersionedMigration<
0,
1,
version_unchecked::MigrateV0ToV1<T, UnlockParaIds>,
super::Pallet<T>,
<T as frame_system::Config>::DbWeight,
>;
}
}
4 changes: 2 additions & 2 deletions polkadot/runtime/kusama/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1727,7 +1727,7 @@ pub mod migrations {
/// Unreleased migrations. Add new ones here:
pub type Unreleased = (
init_state_migration::InitMigrate,
pallet_society::migrations::VersionCheckedMigrateToV2<
pallet_society::migrations::v2::versioned::MigrateV0ToV2<
Runtime,
(),
past_payouts::PastPayouts,
Expand Down Expand Up @@ -1756,7 +1756,7 @@ pub mod migrations {

parachains_configuration::migration::v9::MigrateToV9<Runtime>,
// Migrate parachain info format
paras_registrar::migration::VersionCheckedMigrateToV1<Runtime, ParachainsToUnlock>,
paras_registrar::migration::v1::versioned::MigrateV0ToV1<Runtime, ParachainsToUnlock>,
);
}

Expand Down
2 changes: 1 addition & 1 deletion polkadot/runtime/kusama/src/past_payouts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

use super::*;
use hex_literal::hex;
use pallet_society::migrations::from_raw_past_payouts;
use pallet_society::migrations::v2::from_raw_past_payouts;

parameter_types! {
pub PastPayouts: Vec<(AccountId, Balance)> = from_raw_past_payouts::<Runtime, ()>(vec![
Expand Down
2 changes: 1 addition & 1 deletion polkadot/runtime/polkadot/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1540,7 +1540,7 @@ pub mod migrations {

parachains_configuration::migration::v9::MigrateToV9<Runtime>,
// Migrate parachain info format
paras_registrar::migration::VersionCheckedMigrateToV1<Runtime, ParachainsToUnlock>,
paras_registrar::migration::v1::versioned::MigrateV0ToV1<Runtime, ParachainsToUnlock>,
);
}

Expand Down
6 changes: 3 additions & 3 deletions polkadot/runtime/rococo/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1544,14 +1544,14 @@ pub mod migrations {

/// Unreleased migrations. Add new ones here:
pub type Unreleased = (
pallet_society::migrations::VersionCheckedMigrateToV2<Runtime, (), ()>,
pallet_society::migrations::v2::versioned::MigrateV0ToV2<Runtime, (), ()>,
pallet_im_online::migration::v1::Migration<Runtime>,
parachains_configuration::migration::v7::MigrateToV7<Runtime>,
assigned_slots::migration::v1::VersionCheckedMigrateToV1<Runtime>,
assigned_slots::migration::v1::versioned::MigrateV0ToV1<Runtime>,
parachains_scheduler::migration::v1::MigrateToV1<Runtime>,
parachains_configuration::migration::v8::MigrateToV8<Runtime>,
parachains_configuration::migration::v9::MigrateToV9<Runtime>,
paras_registrar::migration::VersionCheckedMigrateToV1<Runtime, ()>,
paras_registrar::migration::v1::versioned::MigrateV0ToV1<Runtime, ()>,
);
}

Expand Down
4 changes: 2 additions & 2 deletions polkadot/runtime/westend/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1421,12 +1421,12 @@ pub mod migrations {
pub type Unreleased = (
pallet_im_online::migration::v1::Migration<Runtime>,
parachains_configuration::migration::v7::MigrateToV7<Runtime>,
assigned_slots::migration::v1::VersionCheckedMigrateToV1<Runtime>,
assigned_slots::migration::v1::versioned::MigrateV0ToV1<Runtime>,
parachains_scheduler::migration::v1::MigrateToV1<Runtime>,
parachains_configuration::migration::v8::MigrateToV8<Runtime>,
UpgradeSessionKeys,
parachains_configuration::migration::v9::MigrateToV9<Runtime>,
paras_registrar::migration::VersionCheckedMigrateToV1<Runtime, ()>,
paras_registrar::migration::v1::versioned::MigrateV0ToV1<Runtime, ()>,
);
}

Expand Down
Loading