Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(zk_toolbox): add zki ecosystem build subcommand #2787

Merged
merged 97 commits into from
Sep 16, 2024
Merged
Show file tree
Hide file tree
Changes from 91 commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
402a69f
feat: add `zki ecosystem build` (based on `zki ecosystem init`)
manuelmauro Sep 3, 2024
d5b42cb
refactor: remove chains initialization step
manuelmauro Sep 3, 2024
243973c
refactor: remove dev and observability args, add sender
manuelmauro Sep 3, 2024
bc39ffa
refactor: format code
manuelmauro Sep 3, 2024
6cb1249
refactor: remove l1_rpc_url option from ecosystem build
manuelmauro Sep 3, 2024
110b87b
fix: use dummy RPC endpoint
manuelmauro Sep 3, 2024
f518705
feat: wire forge sender option
manuelmauro Sep 3, 2024
af8f9cb
refactor: remove Paymaster and ERC20 contracts from build subcommand
manuelmauro Sep 3, 2024
c38ba3b
refactor: do not broadcast transactions on build
manuelmauro Sep 3, 2024
c05dffe
feat: create command output directory
manuelmauro Sep 4, 2024
0c64d2b
refactor: rename deploy_ecosystem to build_ecosystem
manuelmauro Sep 4, 2024
76cd0a5
refactor: use more idiomatic flag implementation
manuelmauro Sep 4, 2024
919705e
refactor: move ecosystem contract prompt to args/build
manuelmauro Sep 4, 2024
0c95591
Merge branch 'main' into manuel-add-zki-ecosystem-build-subcommand
manuelmauro Sep 4, 2024
e438a8e
fix: use string constants for logging
manuelmauro Sep 4, 2024
168a92b
feat: add l1_rpc_url command option
manuelmauro Sep 4, 2024
4394d72
refactor: remove useless code
manuelmauro Sep 4, 2024
a01ee90
refactor: remove unneeded mut modifier
manuelmauro Sep 4, 2024
89a3fa9
refactor: absorb init function into run
manuelmauro Sep 4, 2024
3654de7
refactor: merge build_ecosystem and build_ecosystem_inner
manuelmauro Sep 4, 2024
6247231
refactor: remove genesis args
manuelmauro Sep 4, 2024
9f8cfd8
refactor: use single arguments struct
manuelmauro Sep 4, 2024
07d4813
fix: build required config files
manuelmauro Sep 4, 2024
9fea58d
fix: unsigned transactions path
manuelmauro Sep 4, 2024
80c7ffc
refactor: lint
manuelmauro Sep 4, 2024
7aee591
refactor: use PathBuf::join
manuelmauro Sep 4, 2024
30feef0
refactor: remove balance check
manuelmauro Sep 5, 2024
d669fa2
refactor: remove unused imports
manuelmauro Sep 5, 2024
cae36e0
refactor: use positional arg for sender
manuelmauro Sep 5, 2024
db48f49
feat: save deployment config file
manuelmauro Sep 5, 2024
45a9202
feat: use dedicated script params for ecosystem build
manuelmauro Sep 5, 2024
bd87144
feat: draft `zki chain build` subcommand
manuelmauro Sep 5, 2024
78e8d08
refactor: format
manuelmauro Sep 6, 2024
db047fe
feat: first version of zki chain build
manuelmauro Sep 6, 2024
be328a2
fix: use original script params
manuelmauro Sep 6, 2024
20e0492
Merge branch 'main' into manuel-add-zki-ecosystem-build-subcommand
manuelmauro Sep 6, 2024
258eb0c
refactor: use message const
manuelmauro Sep 6, 2024
e9cdc32
refactor: use PathBuf for --out option
manuelmauro Sep 6, 2024
5b0758c
refactor: remove unused option
manuelmauro Sep 6, 2024
b7cce7b
fix: remove unsupported option
manuelmauro Sep 6, 2024
97d7368
refactor: use shadowing for final args
manuelmauro Sep 6, 2024
25b4617
refactor: use message const for error context
manuelmauro Sep 6, 2024
f93c0c7
refactor: add default for L1 RPC address
manuelmauro Sep 6, 2024
9714109
refactor: improve log message
manuelmauro Sep 6, 2024
f31adce
feat: save contracts config to output folder
manuelmauro Sep 6, 2024
7bb2ce8
feat: improve out dir structure
manuelmauro Sep 6, 2024
f7456c8
feat: improve out file naming
manuelmauro Sep 6, 2024
5e8ead3
style: improve out file naming
manuelmauro Sep 6, 2024
9b8aaed
refactor: use const for file names
manuelmauro Sep 6, 2024
e446d7d
refactor: lint
manuelmauro Sep 6, 2024
6a4d039
feat: improve logging
manuelmauro Sep 6, 2024
0cade20
Merge branch 'main' into manuel-add-zki-ecosystem-build-subcommand
manuelmauro Sep 6, 2024
9e19706
refactor: rename `zki ecosystem build` to `zki ecosystem transaction`
manuelmauro Sep 6, 2024
66f07e1
refactor: rename `zki chain build` to `zki chain transaction`
manuelmauro Sep 6, 2024
53923d4
refactor: rename `zki ecosystem transaction` to `zki ecosystem build-…
manuelmauro Sep 9, 2024
c615dbd
refactor: rename `zki chain transaction` to `zki chain build-transact…
manuelmauro Sep 9, 2024
3b78e5f
fix: use consistent args naming
manuelmauro Sep 9, 2024
d4d8359
feat: prompt for sender address
manuelmauro Sep 9, 2024
a4baf97
Merge branch 'main' into manuel-add-zki-ecosystem-build-subcommand
manuelmauro Sep 9, 2024
6fb8ebf
feat: save contracts.yaml to transaction output
manuelmauro Sep 9, 2024
13d34dd
feat: draft send_transactions subcommand
manuelmauro Sep 9, 2024
0c0804d
refactor: use common::Cmd
manuelmauro Sep 9, 2024
4d8ac15
style: format
manuelmauro Sep 9, 2024
7af8532
refactor: use ethers.rs to send transactions
manuelmauro Sep 10, 2024
a170d21
feat: log transaction receipts
manuelmauro Sep 10, 2024
c30f417
style: reorder imports
manuelmauro Sep 10, 2024
3352e63
style: format
manuelmauro Sep 10, 2024
b231bce
refactor: move send-transactions from zki to zks
manuelmauro Sep 10, 2024
29f9591
style: lint use statements
manuelmauro Sep 10, 2024
97444df
refactor: use message constants
manuelmauro Sep 10, 2024
02dfc63
refactor: remove register chain's duplicated code
manuelmauro Sep 11, 2024
a289ef9
fix: remove unused import
manuelmauro Sep 11, 2024
9f8de09
style: format
manuelmauro Sep 11, 2024
30976c3
style: format
manuelmauro Sep 11, 2024
9ddb34d
refactor: remove duplicated code between ecosystem init and build-con…
manuelmauro Sep 11, 2024
8d08bd7
fix: check balance only on broadcast
manuelmauro Sep 11, 2024
d1859bb
style: clippy
manuelmauro Sep 12, 2024
0a2bf54
fix: check balance order
manuelmauro Sep 12, 2024
211219f
fix: send register transaction calls
manuelmauro Sep 12, 2024
6f93b21
Merge branch 'main' into manuel-add-zki-ecosystem-build-subcommand
manuelmauro Sep 12, 2024
c821508
style: format code
manuelmauro Sep 12, 2024
66a7a1f
feat: do not prompt for gas price
manuelmauro Sep 13, 2024
d350edf
feat: add outro message to send-transactions
manuelmauro Sep 13, 2024
b627bc2
refactor: use function message instead of const
manuelmauro Sep 13, 2024
e3ae8be
refactor: do not prompt for transaction confirmations
manuelmauro Sep 13, 2024
5c92da2
fix: source chain_id from chain_config instead of genesis
manuelmauro Sep 13, 2024
6191ead
ci: test offline chain registration
manuelmauro Sep 13, 2024
31121d3
Merge branch 'main' into manuel-add-zki-ecosystem-build-subcommand
manuelmauro Sep 13, 2024
1ecc532
style: lint
manuelmauro Sep 13, 2024
1c9c2ee
fix: prefix cast with ci_run
manuelmauro Sep 13, 2024
2915f9f
style: add EOF newline
manuelmauro Sep 16, 2024
875adef
fix: add basic retry mechanism to send-transactions
manuelmauro Sep 16, 2024
94aa940
Merge branch 'main' into manuel-add-zki-ecosystem-build-subcommand
manuelmauro Sep 16, 2024
311c35e
fix: remove duplicated dependency
manuelmauro Sep 16, 2024
94f9180
fix: log message
manuelmauro Sep 16, 2024
45dbff4
refactor: log error to terminal
manuelmauro Sep 16, 2024
fc400db
Merge branch 'main' into manuel-add-zki-ecosystem-build-subcommand
manuelmauro Sep 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 35 additions & 3 deletions .github/workflows/ci-zk-toolbox-reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ jobs:
submodules: "recursive"
fetch-depth: 0


- name: Setup environment
run: |
echo ZKSYNC_HOME=$(pwd) >> $GITHUB_ENV
Expand Down Expand Up @@ -79,6 +78,41 @@ jobs:
--ignore-prerequisites --verbose \
--observability=false

- name: Create and register chain with transactions signed "offline"
run: |
ci_run zk_inception chain create \
--chain-name offline_chain \
--chain-id sequential \
--prover-mode no-proofs \
--wallet-creation localhost \
--l1-batch-commit-data-generator-mode rollup \
--base-token-address 0x0000000000000000000000000000000000000001 \
--base-token-price-nominator 1 \
--base-token-price-denominator 1 \
--set-as-default false \
--ignore-prerequisites

ci_run zk_inception chain build-transactions --chain offline_chain --l1-rpc-url http://127.0.0.1:8545

governor_pk=$(awk '/governor:/ {flag=1} flag && /private_key:/ {print $2; exit}' ./configs/wallets.yaml)

ci_run zk_supervisor send-transactions \
--file ./transactions/chain/offline_chain/register-hyperchain-txns.json \
--l1-rpc-url http://127.0.0.1:8545 \
--private-key $governor_pk

bridge_hub=$(awk '/bridgehub_proxy_addr/ {print $2}' ./configs/contracts.yaml)
chain_id=$(awk '/chain_id:/ {print $2}' ./chains/offline_chain/ZkStack.yaml)

hyperchain_output=$(ci_run cast call $bridge_hub "getHyperchain(uint256)" $chain_id)

if [[ $hyperchain_output == 0x* && ${#hyperchain_output} -eq 66 ]]; then
echo "Chain successfully registered: $hyperchain_output"
else
echo "Failed to register chain: $hyperchain_output"
exit 1
fi

- name: Read Custom Token address and set as environment variable
run: |
address=$(awk -F": " '/tokens:/ {found_tokens=1} found_tokens && /DAI:/ {found_dai=1} found_dai && /address:/ {print $2; exit}' ./configs/erc20.yaml)
Expand Down Expand Up @@ -297,15 +331,13 @@ jobs:
wait $PID3
wait $PID4


# Upgrade tests should run last, because as soon as they
# finish the bootloader will be different
# TODO make upgrade tests safe to run multiple times
- name: Run upgrade test
run: |
ci_run zk_supervisor test upgrade --no-deps --chain era


- name: Upload logs
uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
if: always()
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -117,3 +117,4 @@ chains/era/configs/*
configs/*
era-observability/
core/tests/ts-integration/deployments-zk
transactions/
6 changes: 3 additions & 3 deletions infrastructure/zk/src/docker.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import {Command} from 'commander';
import { Command } from 'commander';
import * as utils from 'utils';

const IMAGES = [
Expand Down Expand Up @@ -31,7 +31,7 @@ async function dockerCommand(
dockerOrg: string = 'matterlabs'
) {
// Generating all tags for containers. We need 2 tags here: SHA and SHA+TS
const {stdout: COMMIT_SHORT_SHA}: { stdout: string } = await utils.exec('git rev-parse --short HEAD');
const { stdout: COMMIT_SHORT_SHA }: { stdout: string } = await utils.exec('git rev-parse --short HEAD');
// COMMIT_SHORT_SHA returns with newline, so we need to trim it
const imageTagShaTS: string = process.env.IMAGE_TAG_SUFFIX
? process.env.IMAGE_TAG_SUFFIX
Expand Down Expand Up @@ -126,7 +126,7 @@ async function _build(image: string, tagList: string[], dockerOrg: string, platf
}
buildArgs += extraArgs;

console.log("Build args: ", buildArgs);
console.log('Build args: ', buildArgs);

const buildCommand =
`DOCKER_BUILDKIT=1 docker buildx build ${tagsToBuild}` +
Expand Down
2 changes: 2 additions & 0 deletions zk_toolbox/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions zk_toolbox/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ clap = { version = "4.4", features = ["derive", "wrap_help", "string"] }
slugify-rs = "0.0.3"
cliclack = "0.2.5"
console = "0.15.8"
chrono = "0.4.38"
ethers = "2.0"
futures = "0.3.30"
human-panic = "2.0"
Expand Down
11 changes: 11 additions & 0 deletions zk_toolbox/crates/common/src/forge.rs
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,12 @@ impl ForgeScript {
self
}

/// Add the sender address to the forge script command.
pub fn with_sender(mut self, address: String) -> Self {
self.args.add_arg(ForgeScriptArg::Sender { address });
self
}

/// Add the rpc-url flag to the forge script command.
pub fn with_rpc_url(mut self, rpc_url: String) -> Self {
self.args.add_arg(ForgeScriptArg::RpcUrl { url: rpc_url });
Expand Down Expand Up @@ -135,6 +141,7 @@ impl ForgeScript {
});
self
}

// Do not start the script if balance is not enough
pub fn private_key(&self) -> Option<H256> {
self.args.args.iter().find_map(|a| {
Expand Down Expand Up @@ -244,6 +251,10 @@ pub enum ForgeScriptArg {
},
Verify,
Resume,
#[strum(to_string = "sender={address}")]
Sender {
address: String,
},
}

/// ForgeScriptArgs is a set of arguments that can be passed to the forge script command.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ impl ZkToolboxConfig for RegisterChainL1Config {}

impl RegisterChainL1Config {
pub fn new(chain_config: &ChainConfig, contracts: &ContractsConfig) -> anyhow::Result<Self> {
let genesis_config = chain_config.get_genesis_config()?;
let wallets_config = chain_config.get_wallets_config()?;
Ok(Self {
contracts_config: Contracts {
Expand All @@ -72,7 +71,7 @@ impl RegisterChainL1Config {
validator_timelock_addr: contracts.ecosystem_contracts.validator_timelock_addr,
},
chain: ChainL1Config {
chain_chain_id: genesis_config.l2_chain_id,
chain_chain_id: chain_config.chain_id,
base_token_gas_price_multiplier_nominator: chain_config.base_token.nominator,
base_token_gas_price_multiplier_denominator: chain_config.base_token.denominator,
base_token_addr: chain_config.base_token.address,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
use std::path::PathBuf;

use clap::Parser;
use common::{config::global_config, forge::ForgeScriptArgs, Prompt};
use serde::{Deserialize, Serialize};
use url::Url;

use crate::{
consts::DEFAULT_UNSIGNED_TRANSACTIONS_DIR,
defaults::LOCAL_RPC_URL,
messages::{MSG_L1_RPC_URL_HELP, MSG_L1_RPC_URL_INVALID_ERR, MSG_L1_RPC_URL_PROMPT},
};

const CHAIN_SUBDIR: &str = "chain";

#[derive(Debug, Clone, Serialize, Deserialize, Parser)]
pub struct BuildTransactionsArgs {
/// Output directory for the generated files.
#[arg(long, short)]
pub out: Option<PathBuf>,
/// All ethereum environment related arguments
#[clap(flatten)]
#[serde(flatten)]
pub forge_args: ForgeScriptArgs,
#[clap(long, help = MSG_L1_RPC_URL_HELP)]
pub l1_rpc_url: Option<String>,
}

impl BuildTransactionsArgs {
pub fn fill_values_with_prompt(self, default_chain: String) -> BuildTransactionsArgsFinal {
let chain_name = global_config().chain_name.clone();

let l1_rpc_url = self.l1_rpc_url.unwrap_or_else(|| {
Prompt::new(MSG_L1_RPC_URL_PROMPT)
.default(LOCAL_RPC_URL)
.validate_with(|val: &String| -> Result<(), String> {
Url::parse(val)
.map(|_| ())
.map_err(|_| MSG_L1_RPC_URL_INVALID_ERR.to_string())
})
.ask()
});

BuildTransactionsArgsFinal {
out: self
.out
.unwrap_or(PathBuf::from(DEFAULT_UNSIGNED_TRANSACTIONS_DIR).join(CHAIN_SUBDIR))
.join(chain_name.unwrap_or(default_chain)),
forge_args: self.forge_args,
l1_rpc_url,
}
}
}

#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct BuildTransactionsArgsFinal {
pub out: PathBuf,
pub forge_args: ForgeScriptArgs,
pub l1_rpc_url: String,
}
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
pub mod build_transactions;
pub mod create;
pub mod genesis;
pub mod init;
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
use anyhow::Context;
use common::{config::global_config, git, logger, spinner::Spinner};
use config::{
copy_configs, traits::SaveConfigWithBasePath, update_from_chain_config, EcosystemConfig,
};
use ethers::utils::hex::ToHex;
use xshell::Shell;

use super::common::register_chain;
use crate::{
commands::chain::args::build_transactions::BuildTransactionsArgs,
messages::{
MSG_BUILDING_CHAIN_REGISTRATION_TXNS_SPINNER, MSG_CHAIN_INITIALIZED,
MSG_CHAIN_NOT_FOUND_ERR, MSG_CHAIN_TXN_MISSING_CONTRACT_CONFIG,
MSG_CHAIN_TXN_OUT_PATH_INVALID_ERR, MSG_PREPARING_CONFIG_SPINNER, MSG_SELECTED_CONFIG,
MSG_WRITING_OUTPUT_FILES_SPINNER,
},
};

const REGISTER_CHAIN_TXNS_FILE_SRC: &str =
"contracts/l1-contracts/broadcast/RegisterHyperchain.s.sol/9/dry-run/run-latest.json";
const REGISTER_CHAIN_TXNS_FILE_DST: &str = "register-hyperchain-txns.json";

const SCRIPT_CONFIG_FILE_SRC: &str =
"contracts/l1-contracts/script-config/register-hyperchain.toml";
const SCRIPT_CONFIG_FILE_DST: &str = "register-hyperchain.toml";

pub(crate) async fn run(args: BuildTransactionsArgs, shell: &Shell) -> anyhow::Result<()> {
let config = EcosystemConfig::from_file(shell)?;
let chain_name = global_config().chain_name.clone();
let chain_config = config
.load_chain(chain_name)
.context(MSG_CHAIN_NOT_FOUND_ERR)?;

let args = args.fill_values_with_prompt(config.default_chain.clone());

git::submodule_update(shell, config.link_to_code.clone())?;

let spinner = Spinner::new(MSG_PREPARING_CONFIG_SPINNER);
copy_configs(shell, &config.link_to_code, &chain_config.configs)?;

logger::note(MSG_SELECTED_CONFIG, logger::object_to_string(&chain_config));

let mut genesis_config = chain_config.get_genesis_config()?;
update_from_chain_config(&mut genesis_config, &chain_config);

// Copy ecosystem contracts
let mut contracts_config = config
.get_contracts_config()
.context(MSG_CHAIN_TXN_MISSING_CONTRACT_CONFIG)?;
contracts_config.l1.base_token_addr = chain_config.base_token.address;
spinner.finish();

let spinner = Spinner::new(MSG_BUILDING_CHAIN_REGISTRATION_TXNS_SPINNER);
let governor: String = config.get_wallets()?.governor.address.encode_hex_upper();

register_chain(
shell,
args.forge_args.clone(),
&config,
&chain_config,
&mut contracts_config,
args.l1_rpc_url.clone(),
Some(governor),
false,
)
.await?;

contracts_config.save_with_base_path(shell, &args.out)?;
spinner.finish();

let spinner = Spinner::new(MSG_WRITING_OUTPUT_FILES_SPINNER);
shell
.create_dir(&args.out)
.context(MSG_CHAIN_TXN_OUT_PATH_INVALID_ERR)?;

shell.copy_file(
config.link_to_code.join(REGISTER_CHAIN_TXNS_FILE_SRC),
args.out.join(REGISTER_CHAIN_TXNS_FILE_DST),
)?;

shell.copy_file(
config.link_to_code.join(SCRIPT_CONFIG_FILE_SRC),
args.out.join(SCRIPT_CONFIG_FILE_DST),
)?;
spinner.finish();

logger::success(MSG_CHAIN_INITIALIZED);
Ok(())
}
Loading
Loading