Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Provide easy prover setup #2683

Merged
merged 36 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
a5e858c
make prover run without core
Artemka374 Aug 19, 2024
863e601
fix build
Artemka374 Aug 19, 2024
9c85865
add docs
Artemka374 Aug 19, 2024
7529c3f
fix build, address comments
Artemka374 Aug 19, 2024
9078100
fix build
Artemka374 Aug 19, 2024
6e7be8d
add some prover_cli commands
Artemka374 Aug 19, 2024
03832b1
address comments
Artemka374 Aug 20, 2024
dd9bd7a
add protocol-version command, update docs
Artemka374 Aug 20, 2024
b58d2d5
fix build
Artemka374 Aug 20, 2024
752cd01
update some commands
Artemka374 Aug 20, 2024
c0249cf
Merge branch 'refs/heads/main' into afo/prover-setup-2.0
Artemka374 Aug 21, 2024
9274787
fix problems with keystore
Artemka374 Aug 21, 2024
a655222
debug
Artemka374 Aug 21, 2024
cd811ee
debug
Artemka374 Aug 21, 2024
56d385b
fix path of generate-sk
Artemka374 Aug 21, 2024
32b5bfc
fix?
Artemka374 Aug 21, 2024
d4f7063
update docs
Artemka374 Aug 21, 2024
bd7709c
Merge branch 'main' into afo/prover-setup-2.0
Artemka374 Aug 21, 2024
3b45aba
fix build, fix lint
Artemka374 Aug 21, 2024
f254d99
fmt
Artemka374 Aug 21, 2024
5bab4c9
Merge remote-tracking branch 'origin/afo/prover-setup-2.0' into afo/p…
Artemka374 Aug 21, 2024
db4b0f9
update doc
Artemka374 Aug 21, 2024
4e3ffb6
fix test
Artemka374 Aug 21, 2024
374118b
fix generate-sk
Artemka374 Aug 21, 2024
25567e4
add pjm to prover components
Artemka374 Aug 21, 2024
b2c3639
fix name
Artemka374 Aug 21, 2024
c412a32
add pjm to doc
Artemka374 Aug 21, 2024
8e15375
update docs
Artemka374 Aug 22, 2024
e8c8617
update docs
Artemka374 Aug 22, 2024
95b2e36
Merge branch 'main' into afo/prover-setup-2.0
Artemka374 Aug 22, 2024
6ad0f21
address comments
Artemka374 Aug 22, 2024
5a30a2d
Merge remote-tracking branch 'origin/afo/prover-setup-2.0' into afo/p…
Artemka374 Aug 22, 2024
fda7e4d
update docs
Artemka374 Aug 22, 2024
1dc3c35
fix lint
Artemka374 Aug 22, 2024
ecedbe3
fmt
Artemka374 Aug 22, 2024
eae9818
rename to prover-version
Artemka374 Aug 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions prover/docs/05_proving_batch.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,14 @@ input file, called `witness_inputs_<batch>.bin` generated by different core comp
- If you want to prove the batch produced by zkSync, you can get the data from the `ExternalProofIntegrationAPI` using
Artemka374 marked this conversation as resolved.
Show resolved Hide resolved
`{address}/proof_generation_data` endpoint. You need to replace `{address}` with the address of the API and provide
the batch number as a query data to get the data for specific batch, otherwise, you will receive latest data for the
batch, that was already proven.
batch, that was already proven. Example:
```shell
curl -H "Content-Type: application/json" -X POST {address}/proof_generation_data -d 'null'
```
or
```shell
curl -H "Content-Type: application/json" -X POST {address}/proof_generation_data -d '1000'
```

### Preparing database

Expand All @@ -80,12 +87,19 @@ the protocol version it should use. You can do that with running
zk_inception prover protocol-version
Artemka374 marked this conversation as resolved.
Show resolved Hide resolved
Artemka374 marked this conversation as resolved.
Show resolved Hide resolved
```

Example output:

```shell
Current protocol version found in zksync-era: 0.24.2, snark_wrapper: "0x14f97b81e54b35fe673d8708cc1a19e1ea5b5e348e12d31e39824ed4f42bbca2"
```

This command will provide you with the information about the semantic protocol version(you need to know only minor and
patch versions) and snark wrapper value.
patch versions) and snark wrapper value. In the example, `MINOR_VERSION` is 24, `PATCH_VERSION` is 2, and
`SNARK_WRAPPER` is `0x14f97b81e54b35fe673d8708cc1a19e1ea5b5e348e12d31e39824ed4f42bbca2`.

Now, with the use of `prover_cli` tool, you can insert the data about the batch and protocol version into the database:

First, get the database URL(you can find it in `<ecosystem_dir>/chains/configs/<chain_name>/secrets.yaml` - it is the
First, get the database URL(you can find it in `<ecosystem_dir>/chains/<chain_name>/configs/secrets.yaml` - it is the
`prover_url` value) Now, insert the information about protocol version in the database:

```shell
Expand Down
1 change: 1 addition & 0 deletions zk_toolbox/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 0 additions & 3 deletions zk_toolbox/crates/zk_inception/src/commands/prover/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ mod gcs;
mod generate_sk;
mod init;
mod init_bellman_cuda;
mod protocol_version;
mod run;
mod utils;

Expand All @@ -23,7 +22,6 @@ pub enum ProverCommands {
/// Initialize bellman-cuda
#[command(alias = "cuda")]
InitBellmanCuda(Box<InitBellmanCudaArgs>),
ProtocolVersion,
}

pub(crate) async fn run(shell: &Shell, args: ProverCommands) -> anyhow::Result<()> {
Expand All @@ -32,6 +30,5 @@ pub(crate) async fn run(shell: &Shell, args: ProverCommands) -> anyhow::Result<(
ProverCommands::GenerateSK => generate_sk::run(shell).await,
ProverCommands::Run(args) => run::run(args, shell).await,
ProverCommands::InitBellmanCuda(args) => init_bellman_cuda::run(shell, *args).await,
ProverCommands::ProtocolVersion => protocol_version::run(shell).await,
}
}
1 change: 1 addition & 0 deletions zk_toolbox/crates/zk_supervisor/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,4 @@ xshell.workspace = true
serde.workspace = true
clap-markdown.workspace = true
futures.workspace = true
serde_json.workspace = true
1 change: 1 addition & 0 deletions zk_toolbox/crates/zk_supervisor/src/commands/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@ pub mod database;
pub mod fmt;
pub mod lint;
pub(crate) mod lint_utils;
pub mod protocol_version;
pub mod snapshot;
pub mod test;
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ use xshell::{cmd, Shell};

pub async fn run(shell: &Shell) -> anyhow::Result<()> {
let link_to_code = EcosystemConfig::from_file(shell)?.link_to_code;
let link_to_prover = link_to_code.join("prover");

let protocol_version = get_protocol_version(shell, &link_to_code).await?;
let snark_wrapper = get_snark_wrapper(&link_to_code).await?;
let protocol_version = get_protocol_version(shell, &link_to_prover).await?;
let snark_wrapper = get_snark_wrapper(&link_to_prover).await?;

logger::info(format!(
"Current protocol version found in zksync-era: {}, snark_wrapper: {}",
Expand All @@ -18,20 +19,16 @@ pub async fn run(shell: &Shell) -> anyhow::Result<()> {
Ok(())
}

async fn get_protocol_version(shell: &Shell, link_to_code: &Path) -> anyhow::Result<String> {
let path = link_to_code.join("prover/crates/bin/prover_version/Cargo.toml");
let protocol_version = cmd!(
shell,
"cargo +nightly-2024-08-01 run --manifest-path {path}"
)
.read()?;
async fn get_protocol_version(shell: &Shell, link_to_prover: &Path) -> anyhow::Result<String> {
shell.change_dir(link_to_prover);
let protocol_version = cmd!(shell, "cargo run --release --bin prover_version").read()?;

Ok(protocol_version)
}

async fn get_snark_wrapper(link_to_code: &Path) -> anyhow::Result<String> {
let path = link_to_code
.join("prover/crates/bin/vk_setup_data_generator_server_fri/data/commitments.json");
async fn get_snark_wrapper(link_to_prover: &Path) -> anyhow::Result<String> {
let path =
link_to_prover.join("crates/bin/vk_setup_data_generator_server_fri/data/commitments.json");
let file = fs::File::open(path).expect("Could not find commitments file in zksync-era");
let json: serde_json::Value =
serde_json::from_reader(file).expect("Could not parse commitments.json");
Expand Down
9 changes: 6 additions & 3 deletions zk_toolbox/crates/zk_supervisor/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ use common::{
};
use config::EcosystemConfig;
use messages::{
msg_global_chain_does_not_exist, MSG_SUBCOMMAND_CLEAN, MSG_SUBCOMMAND_DATABASE_ABOUT,
MSG_SUBCOMMAND_FMT_ABOUT, MSG_SUBCOMMAND_LINT_ABOUT, MSG_SUBCOMMAND_SNAPSHOTS_CREATOR_ABOUT,
MSG_SUBCOMMAND_TESTS_ABOUT,
msg_global_chain_does_not_exist, MSG_PROTOCOL_VERSION_ABOUT, MSG_SUBCOMMAND_CLEAN,
MSG_SUBCOMMAND_DATABASE_ABOUT, MSG_SUBCOMMAND_FMT_ABOUT, MSG_SUBCOMMAND_LINT_ABOUT,
MSG_SUBCOMMAND_SNAPSHOTS_CREATOR_ABOUT, MSG_SUBCOMMAND_TESTS_ABOUT,
};
use xshell::Shell;

Expand Down Expand Up @@ -47,6 +47,8 @@ enum SupervisorSubcommands {
Fmt(FmtArgs),
#[command(hide = true)]
Markdown,
#[command(about = MSG_PROTOCOL_VERSION_ABOUT)]
ProtocolVersion,
}

#[derive(Parser, Debug)]
Expand Down Expand Up @@ -103,6 +105,7 @@ async fn run_subcommand(args: Supervisor, shell: &Shell) -> anyhow::Result<()> {
}
SupervisorSubcommands::Lint(args) => commands::lint::run(shell, args)?,
SupervisorSubcommands::Fmt(args) => commands::fmt::run(shell.clone(), args).await?,
SupervisorSubcommands::ProtocolVersion => commands::protocol_version::run(shell).await?,
Artemka374 marked this conversation as resolved.
Show resolved Hide resolved
}
Ok(())
}
Expand Down
1 change: 1 addition & 0 deletions zk_toolbox/crates/zk_supervisor/src/messages.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ pub(super) fn msg_global_chain_does_not_exist(chain: &str, available_chains: &st
}

// Subcommands help
pub(super) const MSG_PROTOCOL_VERSION_ABOUT: &str = "Protocol version used by provers";
pub(super) const MSG_SUBCOMMAND_DATABASE_ABOUT: &str = "Database related commands";
pub(super) const MSG_SUBCOMMAND_TESTS_ABOUT: &str = "Run tests";
pub(super) const MSG_SUBCOMMAND_CLEAN: &str = "Clean artifacts";
Expand Down