Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Typed Storage Keys #1419

Merged
merged 49 commits into from
Mar 6, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
eeded2c
first iteration on storage multi keys
tadeohepperle Feb 8, 2024
4f052d0
decoding values from concat style hashers
tadeohepperle Feb 8, 2024
e4e4437
move util functions and remove comments
tadeohepperle Feb 8, 2024
3f20d77
change codegen for storage keys and fix examples
tadeohepperle Feb 9, 2024
3f0d8f0
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Feb 9, 2024
28e1b77
trait bounds don't match scale value...
tadeohepperle Feb 13, 2024
489ccdc
fix trait bounds and examples
tadeohepperle Feb 13, 2024
23203da
reconstruct storage keys in iterations
tadeohepperle Feb 13, 2024
7a36ce3
build(deps): bump js-sys from 0.3.67 to 0.3.68 (#1428)
dependabot[bot] Feb 12, 2024
d7c658d
build(deps): bump clap from 4.4.18 to 4.5.0 (#1427)
dependabot[bot] Feb 12, 2024
b21e94d
build(deps): bump either from 1.9.0 to 1.10.0 (#1425)
dependabot[bot] Feb 12, 2024
252e31e
build(deps): bump thiserror from 1.0.56 to 1.0.57 (#1424)
dependabot[bot] Feb 12, 2024
db12bd7
build(deps): bump jsonrpsee from 0.21.0 to 0.22.0 (#1426)
dependabot[bot] Feb 12, 2024
5313f6a
subxt: Derive `std::cmp` traits for subxt payloads and addresses (#1429)
lexnv Feb 13, 2024
8e1f77f
Merge
tadeohepperle Feb 13, 2024
a2ad8d1
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Feb 14, 2024
cef1f6b
fix clippy
tadeohepperle Feb 15, 2024
62fd01f
add integration tests
tadeohepperle Feb 15, 2024
394b206
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Feb 15, 2024
b131b0f
fix doc tests
tadeohepperle Feb 16, 2024
1b856bc
change hashing logic for hashers=1
tadeohepperle Feb 16, 2024
b0ddafa
refactor
tadeohepperle Feb 16, 2024
3202601
clippy and fmt
tadeohepperle Feb 19, 2024
0b4f2ea
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Feb 19, 2024
4c750f4
regenerate polkadot file which got changed by the automatic PR
tadeohepperle Feb 19, 2024
a404c85
nested design for storage keys
tadeohepperle Feb 23, 2024
e87eb3c
refactor codegen
tadeohepperle Feb 26, 2024
9c27b47
codegen adjustments
tadeohepperle Feb 26, 2024
585c01c
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Feb 26, 2024
4bfaa4c
fix storage hasher codegen test
tadeohepperle Feb 27, 2024
bd01aab
Suggestions for storage value decoding (#1457)
jsdw Feb 28, 2024
a284ed8
integrate nits from PR
tadeohepperle Feb 28, 2024
5a68e8e
add fuzztest for storage keys, fix decoding bug
tadeohepperle Feb 28, 2024
e4195b2
clippy and fmt
tadeohepperle Feb 28, 2024
97962ee
clippy
tadeohepperle Feb 29, 2024
8f0eb04
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Feb 29, 2024
9b33cbc
Niklas Suggestions
tadeohepperle Feb 29, 2024
b7a6b41
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Mar 1, 2024
aa5e703
lifetime issues and iterator impls
tadeohepperle Mar 4, 2024
1eaa75d
fmt and clippy
tadeohepperle Mar 4, 2024
56f1864
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Mar 4, 2024
c5e071d
regenerate polkadot.rs
tadeohepperle Mar 4, 2024
4a3b936
fix storage key encoding for empty keys
tadeohepperle Mar 5, 2024
a660a67
Merge branch 'master' into tadeohepperle/decoding-storage-keys
tadeohepperle Mar 5, 2024
f7a530c
rename trait methods for storage keys
tadeohepperle Mar 5, 2024
26947b5
fix hasher bug...
tadeohepperle Mar 5, 2024
81cd16e
impl nits, add iterator struct seperate from `StorageHashers`
tadeohepperle Mar 6, 2024
1a42c9e
clippy fix
tadeohepperle Mar 6, 2024
809331e
remove println
tadeohepperle Mar 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 20 additions & 3 deletions codegen/src/api/storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,24 @@ fn generate_storage_entry_fns(
};
let is_fetchable_type = is_fetchable.then_some(quote!(#crate_path::storage::address::Yes)).unwrap_or(quote!(()));
let is_iterable_type = is_iterable.then_some(quote!(#crate_path::storage::address::Yes)).unwrap_or(quote!(()));
let key_impls = keys_slice.iter().map(|(field_name, _, _)| quote!( #crate_path::storage::address::make_static_storage_map_key(#field_name.borrow()) ));
let (keys, keys_type) = match keys_slice.len(){
0 => (quote!( () ), quote!( () )),
1 => {
let field_name = &keys_slice[0].0;
let keys = quote!( #crate_path::storage::address::StorageKey::new(#field_name.borrow()) );
let path = &keys_slice[0].2;
let path = quote!( #crate_path::storage::address::StorageKey<#path> );
(keys, path)
}
_ => {
let keys_iter = keys_slice.iter().map(|(field_name, _, _)| quote!( #crate_path::storage::address::StorageKey::new(#field_name.borrow()) ));
let keys = quote!( (#(#keys_iter,)*) );
let paths_iter = keys_slice.iter().map(|(_, _, path_to_alias)| quote!( #crate_path::storage::address::StorageKey<#path_to_alias> ) );
let paths = quote!( (#(#paths_iter,)*) );
(keys, paths)
}
};

let key_args = keys_slice.iter().map(|(field_name, _, path_to_alias )| {
quote!( #field_name: impl ::std::borrow::Borrow<#path_to_alias> )
});
Expand All @@ -159,7 +176,7 @@ fn generate_storage_entry_fns(
&self,
#(#key_args,)*
) -> #crate_path::storage::address::Address::<
#crate_path::storage::address::StaticStorageMapKey,
#keys_type,
#alias_storage_path,
#is_fetchable_type,
#is_defaultable_type,
Expand All @@ -168,7 +185,7 @@ fn generate_storage_entry_fns(
#crate_path::storage::address::Address::new_static(
#pallet_name,
#storage_name,
vec![#(#key_impls,)*],
#keys,
[#(#storage_hash,)*]
)
}
Expand Down
7 changes: 4 additions & 3 deletions subxt/examples/storage_iterating.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
// a time from the node, but we always iterate over one at a time).
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;

while let Some(Ok((key, value))) = results.next().await {
println!("Key: 0x{}", hex::encode(&key));
println!("Value: {:?}", value);
while let Some(Ok(kv)) = results.next().await {
println!("Keys decoded: {:?}", kv.keys);
println!("Key: 0x{}", hex::encode(&kv.key_bytes));
println!("Value: {:?}", kv.value);
}

Ok(())
Expand Down
11 changes: 6 additions & 5 deletions subxt/examples/storage_iterating_dynamic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,17 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
let api = OnlineClient::<PolkadotConfig>::new().await?;

// Build a dynamic storage query to iterate account information.
// With a dynamic query, we can just provide an empty Vec as the keys to iterate over all entries.
let keys = Vec::<()>::new();
// With a dynamic query, we can just provide an empty vector as the keys to iterate over all entries.
let keys: Vec<scale_value::Value> = vec![];
let storage_query = subxt::dynamic::storage("System", "Account", keys);

// Use that query to return an iterator over the results.
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;

while let Some(Ok((key, value))) = results.next().await {
println!("Key: 0x{}", hex::encode(&key));
println!("Value: {:?}", value.to_value()?);
while let Some(Ok(kv)) = results.next().await {
println!("Keys decoded: {:?}", kv.keys);
println!("Key: 0x{}", hex::encode(&kv.key_bytes));
println!("Value: {:?}", kv.value.to_value()?);
}

Ok(())
Expand Down
8 changes: 4 additions & 4 deletions subxt/examples/storage_iterating_partial.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,11 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Get back an iterator of results.
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;

while let Some(Ok((key, value))) = results.next().await {
println!("Key: 0x{}", hex::encode(&key));
println!("Value: {:?}", value);
while let Some(Ok(kv)) = results.next().await {
println!("Keys decoded: {:?}", kv.keys);
println!("Key: 0x{}", hex::encode(&kv.key_bytes));
println!("Value: {:?}", kv.value);
Comment on lines +42 to +44
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The examples look great!

}

Ok(())
}

Expand Down
3 changes: 3 additions & 0 deletions subxt/src/error/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,9 @@ pub enum StorageAddressError {
/// The number of fields in the metadata for this storage entry.
fields: usize,
},
/// The bytes of a storage address are not the expected address for decoding the storage keys of the address.
#[error("Storage address bytes are not the expected format")]
Copy link
Collaborator

@lexnv lexnv Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a small section about keys needing at least 16 bytes (pallet ++ entry)

Edit: ah this can also mean we are needing 8 bytes and so on (prob good to add a u8 here?)

UnexpectedAddressBytes,
}

/// Something went wrong trying to access details in the metadata.
Expand Down
5 changes: 2 additions & 3 deletions subxt/src/storage/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,13 @@ pub mod utils;

pub use storage_client::StorageClient;

pub use storage_type::Storage;
pub use storage_type::{Storage, StorageKeyValuePair};

/// Types representing an address which describes where a storage
/// entry lives and how to properly decode it.
pub mod address {
pub use super::storage_address::{
dynamic, make_static_storage_map_key, Address, DynamicAddress, StaticStorageMapKey,
StorageAddress, Yes,
dynamic, Address, DynamicAddress, StorageAddress, StorageKey, Yes,
};
}

Expand Down
Loading
Loading