Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite Task of listening Stellar Messages #545

Merged
merged 66 commits into from
Oct 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
cc7c9cc
add logs; change log level
b-yap Jul 5, 2024
5ea8b71
adding started logs for each task
b-yap Jul 5, 2024
9a4b1c4
changing debug logs to info
b-yap Jul 8, 2024
8b75f56
reduce restart interval
b-yap Jul 11, 2024
519b7eb
revert to debug the other logs
b-yap Jul 11, 2024
878470d
make listen() async; use tokio::select
b-yap Jul 30, 2024
67e540c
first iteration
b-yap Aug 1, 2024
12779c7
fix the trait bound `service::Error<error::Error>: From<service::Err…
b-yap Aug 2, 2024
806568d
cleanup clones; add names to threads
b-yap Aug 9, 2024
b110ad1
add a precheck signal
b-yap Aug 12, 2024
fe68519
reduce timeouts; write log every 2 minutes
b-yap Aug 12, 2024
4d3183b
update config and remove precheck when listening for stellar messages
b-yap Aug 13, 2024
53c88de
log every minute
b-yap Aug 13, 2024
8c345e1
cleanup
b-yap Aug 13, 2024
2ad0d7f
reduce timeout by moving it out to `poll_messages_from_stellar()`
b-yap Aug 13, 2024
0cb2927
fix https://github.com/pendulum-chain/spacewalk/actions/runs/10369361…
b-yap Aug 13, 2024
7e891d4
move checking of proof building inside `fn start_oracle_agent()` for …
b-yap Aug 22, 2024
deba3da
rebase
b-yap Sep 4, 2024
970fac4
Merge branch 'main' into rewrite-agent-vault
b-yap Sep 5, 2024
0004c3a
https://github.com/pendulum-chain/spacewalk/actions/runs/10724299988/…
b-yap Sep 9, 2024
a4f2477
await before performing a compare
b-yap Sep 9, 2024
7b45cc5
update test config, and add `ntest` for long-running test cases
b-yap Sep 11, 2024
bcc5903
use ubuntu 20, instead of the latest 22
b-yap Sep 11, 2024
2d664c4
remove tokio spawns on archives; add ntest
b-yap Sep 13, 2024
d5a5adf
switch to ubuntu-latest
b-yap Sep 13, 2024
b8b4387
add ntest to all test cases
b-yap Sep 13, 2024
76fa194
set timeout to 5 minutes
b-yap Sep 13, 2024
cdb10a4
free up disk space
b-yap Sep 16, 2024
59aca69
Use random secret for connecting to overlay in integration tests
ebma Sep 17, 2024
ae50727
Run `cargo +nightly-2024-02-09 fmt --all`
ebma Sep 17, 2024
4777855
Remove ntest timeouts
ebma Sep 17, 2024
413fda1
Add fallback to default stroop fee
ebma Sep 18, 2024
6b2905e
Reduce amount in redeem test
ebma Sep 18, 2024
2cf9075
Fix compilation error
ebma Sep 18, 2024
1335c98
move `is_proof_building_ready` out of oracle agent
b-yap Sep 19, 2024
5b4307c
run release
b-yap Oct 1, 2024
01046af
use `try_recv` instead
b-yap Oct 1, 2024
be7ba9e
use `try_recv` instead
b-yap Oct 1, 2024
ee156ee
Merge remote-tracking branch 'origin/main' into rewrite-agent-vault
b-yap Oct 1, 2024
2eecdae
https://github.com/pendulum-chain/spacewalk/actions/runs/11120883420/…
b-yap Oct 1, 2024
37bc319
https://github.com/pendulum-chain/spacewalk/pull/545#discussion_r1761…
b-yap Oct 1, 2024
5f833d3
clean testing issues
b-yap Oct 2, 2024
c44bac8
fix issues
b-yap Oct 2, 2024
3f03be8
https://github.com/pendulum-chain/spacewalk/actions/runs/11139347479/…
b-yap Oct 3, 2024
6cc05d7
remove env_logger
b-yap Oct 3, 2024
b1800e8
merge from origin/master
b-yap Oct 3, 2024
9cc31b5
fix fmt and clippy issues
b-yap Oct 3, 2024
9d6fe30
fix nightly issues
b-yap Oct 3, 2024
23a1f9b
remove 2nd declaration of agent
b-yap Oct 4, 2024
524342e
https://github.com/pendulum-chain/spacewalk/actions/runs/11175397072/…
b-yap Oct 4, 2024
9b9dfd3
fix timeout issue
b-yap Oct 7, 2024
7b9ddda
remove env_logger
b-yap Oct 8, 2024
6e3f7c3
remove env_logger; move console-subscribe to dev-dependencies; remove…
b-yap Oct 8, 2024
71222e1
reduce changes
b-yap Oct 8, 2024
458ba4f
cargo fmt and clippy
b-yap Oct 8, 2024
3bd0100
https://github.com/pendulum-chain/spacewalk/pull/545#discussion_r1791…
b-yap Oct 8, 2024
e557b44
remove unnecessary dependencies
b-yap Oct 8, 2024
9cad8e4
fix "the `set-output` command is deprecated and will be disabled soon…
b-yap Oct 9, 2024
e7f3e26
https://github.com/pendulum-chain/spacewalk/pull/545#discussion_r1795…
b-yap Oct 11, 2024
0aa969e
address major comments
b-yap Oct 15, 2024
e2b3f02
run cargo fmt
b-yap Oct 15, 2024
c88e317
remove `unwrap()`s
b-yap Oct 16, 2024
8c07a3d
https://github.com/pendulum-chain/spacewalk/pull/545#discussion_r1802…
b-yap Oct 16, 2024
4d40c33
https://github.com/pendulum-chain/spacewalk/actions/runs/11364931112/…
b-yap Oct 21, 2024
636b608
use latest commit version
b-yap Oct 21, 2024
ebb96c4
use `.expect()` since this is only for test feature
b-yap Oct 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/actions/prerequisite-nightly/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ runs:
token: ${{ inputs.token }}

- name: Install Rust Nightly
uses: ./.github/actions/install-rust-nightly
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This github actions is using a deprecated set-output:

The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files.

Replaced with dtolnay

uses: dtolnay/rust-toolchain@nightly
with:
version: ${{ inputs.version }}
toolchain: ${{ inputs.version }}
components: rustfmt, clippy
target: wasm32-unknown-unknown
2 changes: 1 addition & 1 deletion .github/workflows/ci-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
- name: Check Build
run: |
bash ./scripts/cmd-all build check "--release"

- name: Clippy -- Libraries and Binaries
run: |
bash ./scripts/cmd-all clippy "clippy --lib --bins" "--release -- -W clippy::all -A clippy::style -A forgetting_copy_types -A forgetting_references"
Expand Down
80 changes: 72 additions & 8 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion clients/runtime/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,6 @@ testchain-runtime = { package = "spacewalk-runtime-standalone-testnet", path = "
mainnet-runtime = { package = "spacewalk-runtime-standalone-mainnet", path = "../../testchain/runtime/mainnet", optional = true }

# Substrate Stellar Dependencies
substrate-stellar-sdk = { git = "https://github.com/pendulum-chain/substrate-stellar-sdk", branch = "polkadot-v1.1.0" }
wallet = { path = "../wallet" }


Expand Down
57 changes: 32 additions & 25 deletions clients/runtime/src/rpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ use subxt::{
client::OnlineClient,
events::StaticEvent,
rpc_params,
storage::{address::Yes, StorageAddress},
storage::{address::Yes, Storage, StorageAddress},
tx::TxPayload,
utils::Static,
Error as BasicError,
Expand Down Expand Up @@ -163,7 +163,7 @@ impl SpacewalkParachain {
pub aux: ImportedAux,
}

let _head = self.get_finalized_block_hash().await.unwrap();
let _head = self.get_finalized_block_hash().await.expect("failed to get head");

let _: CreatedBlock<Hash> = self
.rpc
Expand All @@ -175,7 +175,7 @@ impl SpacewalkParachain {
/// This function is used in tests to finalize the current block.
#[cfg(feature = "testing-utils")]
pub async fn manual_finalize(&self) {
let head = self.get_finalized_block_hash().await.unwrap();
let head = self.get_finalized_block_hash().await.expect("failed to get head");

let _: bool = self
.rpc
Expand Down Expand Up @@ -271,11 +271,9 @@ impl SpacewalkParachain {
// For getting the nonce, use latest, possibly non-finalized block.
let storage_key = metadata::storage().system().account(&self.account_id);
let on_chain_nonce = self
.api
.storage()
.at_latest()
.get_latest_storage()
.await
.unwrap()
.expect("Failed to get storage from the latest block hash")
.fetch(&storage_key)
.await
.transpose()
Expand All @@ -293,7 +291,7 @@ impl SpacewalkParachain {
where
Address: StorageAddress<IsFetchable = Yes>,
{
Ok(self.api.storage().at_latest().await.unwrap().fetch(&address).await?)
Ok(self.get_latest_storage().await?.fetch(&address).await?)
}

async fn query_finalized_or_error<Address>(
Expand All @@ -313,13 +311,22 @@ impl SpacewalkParachain {
where
Address: StorageAddress<IsFetchable = Yes, IsDefaultable = Yes>,
{
Ok(self.api.storage().at_latest().await.unwrap().fetch_or_default(&address).await?)
Ok(self.get_latest_storage().await?.fetch_or_default(&address).await?)
}

pub async fn get_finalized_block_hash(&self) -> Result<Option<H256>, Error> {
Ok(Some(self.api.backend().latest_finalized_block_ref().await?.hash()))
}

async fn get_latest_storage(
&self,
) -> Result<Storage<SpacewalkRuntime, OnlineClient<SpacewalkRuntime>>, Error> {
self.api.storage().at_latest().await.map_err(|e| {
log::error!("Failed to get storage from the latest block hash");
Error::SubxtRuntimeError(e)
})
}

/// Subscribe to new parachain blocks.
pub async fn on_block<F, R>(&self, on_block: F) -> Result<(), Error>
where
Expand Down Expand Up @@ -427,17 +434,17 @@ impl SpacewalkParachain {
self.api
.tx()
.create_signed_with_nonce(&call, &signer, nonce.into(), Default::default())
.unwrap()
.expect("failed to create a signed extrinsic")
.submit_and_watch()
.await
.unwrap();
.expect("failed to submit extrinsic to chain");

// now call with outdated nonce
let result = self
.api
.tx()
.create_signed_with_nonce(&call, &signer, 0, Default::default())
.unwrap()
.expect("failed to create a signed extrinsic")
.submit_and_watch()
.await;

Expand All @@ -462,17 +469,17 @@ impl SpacewalkParachain {
self.api
.tx()
.create_signed_with_nonce(&call, &signer, nonce.into(), Default::default())
.unwrap()
.expect("failed to create a signed extrinsic")
.submit()
.await
.unwrap();
.expect("failed to submit extrinsic to chain");

// should call with the same nonce
let result = self
.api
.tx()
.create_signed_with_nonce(&call, &signer, nonce.into(), Default::default())
.unwrap()
.expect("failed to create a signed extrinsic")
.submit_and_watch()
.await;

Expand All @@ -499,7 +506,7 @@ pub trait UtilFuncs {
impl UtilFuncs for SpacewalkParachain {
async fn get_current_chain_height(&self) -> Result<u32, Error> {
let height_query = metadata::storage().system().number();
let height = self.api.storage().at_latest().await.unwrap().fetch(&height_query).await?;
let height = self.get_latest_storage().await?.fetch(&height_query).await?;
match height {
Some(height) => Ok(height),
None => Err(Error::BlockNotFound),
Expand All @@ -510,13 +517,13 @@ impl UtilFuncs for SpacewalkParachain {
self.native_currency_id
}

fn is_this_vault(&self, vault_id: &VaultId) -> bool {
&vault_id.account_id == self.get_account_id()
}

fn get_account_id(&self) -> &AccountId {
&self.account_id
}

fn is_this_vault(&self, vault_id: &VaultId) -> bool {
&vault_id.account_id == self.get_account_id()
}
}

#[async_trait]
Expand Down Expand Up @@ -594,7 +601,7 @@ impl VaultRegistryPallet for SpacewalkParachain {
async fn get_all_vaults(&self) -> Result<Vec<SpacewalkVault>, Error> {
let mut vaults = Vec::new();
let key_addr = metadata::storage().vault_registry().vaults_iter();
let mut iter = self.api.storage().at_latest().await.unwrap().iter(key_addr).await?;
let mut iter = self.get_latest_storage().await?.iter(key_addr).await?;
while let Ok((_, account)) = iter.next().await.ok_or(Error::VaultNotFound)? {
if let VaultStatus::Active(..) = account.status {
vaults.push(account);
Expand Down Expand Up @@ -779,7 +786,7 @@ impl CollateralBalancesPallet for SpacewalkParachain {
async fn get_native_balance_for_id(&self, id: &AccountId) -> Result<Balance, Error> {
let query = metadata::storage().system().account(id);

let result = self.api.storage().at_latest().await.unwrap().fetch(&query).await?;
let result = self.get_latest_storage().await?.fetch(&query).await?;
Ok(result.map(|x| x.data.free).unwrap_or_default())
}

Expand All @@ -790,7 +797,7 @@ impl CollateralBalancesPallet for SpacewalkParachain {
) -> Result<Balance, Error> {
let query = metadata::storage().tokens().accounts(&id, &Static(currency_id));

let result = self.api.storage().at_latest().await.unwrap().fetch(&query).await?;
let result = self.get_latest_storage().await?.fetch(&query).await?;
Ok(result.map(|x| x.free).unwrap_or_default())
}

Expand All @@ -805,7 +812,7 @@ impl CollateralBalancesPallet for SpacewalkParachain {
) -> Result<Balance, Error> {
let query = metadata::storage().tokens().accounts(&id, &Static(currency_id));

let result = self.api.storage().at_latest().await.unwrap().fetch(&query).await?;
let result = self.get_latest_storage().await?.fetch(&query).await?;
Ok(result.map(|x| x.reserved).unwrap_or_default())
}

Expand Down Expand Up @@ -1074,7 +1081,7 @@ impl IssuePallet for SpacewalkParachain {
let mut issue_requests = Vec::new();

let key_addr = metadata::storage().issue().issue_requests_iter();
let mut iter = self.api.storage().at_latest().await.unwrap().iter(key_addr).await?;
let mut iter = self.get_latest_storage().await?.iter(key_addr).await?;

while let Some(result) = iter.next().await {
let (issue_id, request) = result?;
Expand Down
15 changes: 5 additions & 10 deletions clients/runtime/src/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ pub use subxt::ext::sp_core::sr25519::Pair as KeyPair;
use subxt::utils::Static;

use crate::{metadata, Config, SpacewalkRuntime, SS58_PREFIX};
use primitives::stellar::PublicKey;

pub type AccountId = subxt::utils::AccountId32;
pub type Address = subxt::ext::sp_runtime::MultiAddress<AccountId, u32>;
Expand All @@ -17,8 +18,6 @@ pub type H256 = subxt::ext::sp_core::H256;
pub type SpacewalkSigner = subxt::tx::PairSigner<SpacewalkRuntime, KeyPair>;
pub type FixedU128 = sp_arithmetic::FixedU128;

pub use substrate_stellar_sdk as stellar;

pub type IssueId = H256;

pub type StellarPublicKeyRaw = [u8; 32];
Expand Down Expand Up @@ -139,19 +138,15 @@ pub mod currency_id {
Asset::AlphaNum4 { code, issuer } => Ok(format!(
"Stellar({:?}:{:?})",
from_utf8(code).unwrap_or_default(),
from_utf8(
stellar::PublicKey::from_binary(*issuer).to_encoding().as_slice()
)
.unwrap_or_default()
from_utf8(PublicKey::from_binary(*issuer).to_encoding().as_slice())
.unwrap_or_default()
)
.replace('\"', "")),
Asset::AlphaNum12 { code, issuer } => Ok(format!(
"Stellar({:?}:{:?})",
from_utf8(code).unwrap_or_default(),
from_utf8(
stellar::PublicKey::from_binary(*issuer).to_encoding().as_slice()
)
.unwrap_or_default()
from_utf8(PublicKey::from_binary(*issuer).to_encoding().as_slice())
.unwrap_or_default()
)
.replace('\"', "")),
},
Expand Down
Loading
Loading