docs: SDK and CLI usage for proving and verifying (#1051)

* docs: SDK and CLI usage for proving and verifying

* docs: overview

* docs: more updates

* Update book/src/advanced-usage/sdk.md

* chore: delete unused

* chore: remove link

* docs: more SDK + CLI stuff

* docs: stdin section

---------

Co-authored-by: Jonathan Wang <[email protected]>

book/src/advanced-usage/sdk.md

@@ -0,0 +1,131 @@
+# Using the SDK
+
+While the CLI provides a convenient way to build, prove, and verify programs, you may want more fine-grained control over the process. The OpenVM Rust SDK allows you to customize various aspects of the workflow programmatically.
+
+For more information on the basic CLI flow, see [Overview of Basic Usage](./overview.md). Writing a guest program is the same as in the CLI.
+
+## Imports and Setup
+
+If you have a guest program and would like to try running the **host program** specified below, you can do so by adding the following imports and setup at the top of the file. You may need to modify the imports and/or the `SomeStruct` struct to match your program.
+
+```rust
+use openvm::{platform::memory::MEM_SIZE, transpiler::elf::Elf};
+use openvm_circuit::arch::instructions::exe::OpenVmExe
+use openvm_circuit::arch::VmExecutor;
+use openvm_sdk::{config::SdkVmConfig, Sdk, StdIn};
+
+let sdk = Sdk;
+
+#[derive(Serialize, Deserialize)]
+pub struct SomeStruct {
+    pub a: u64,
+    pub b: u64,
+}
+```
+
+## Building and Transpiling a Program
+
+The SDK provides lower-level control over the building and transpiling process.
+
+```rust
+// 1. Build the VmConfig with the extensions needed.
+let vm_config = SdkVmConfig::builder()
+    .system(Default::default())
+    .rv32i(Default::default())
+    .io(Default::default())
+    .build();
+
+// 2a. Build the ELF with guest options and a target filter.
+let guest_opts = GuestOptions::default().with_features(vec!["parallel"]);
+let target_filter = TargetFilter::default().with_kind("bin".to_string());
+let elf = sdk.build(guest_opts, "your_path_project_root", &target_filter)?;
+// 2b. Load the ELF from a file
+let elf = Elf::decode("your_path_to_elf", MEM_SIZE as u32)?;
+
+// 3. Transpile the ELF into a VmExe
+let exe = sdk.transpile(elf, vm_config.transpiler())?;
+```
+
+### Using `SdkVmConfig`
+
+The `SdkVmConfig` struct allows you to specify the extensions and system configuration your VM will use. To customize your own configuration, you can use the `SdkVmConfig::builder()` method and set the extensions and system configuration you want.
+
+## Running a Program
+To run your program and see the public value output, you can do the following:
+
+```rust
+// 4. Format your input into StdIn
+let my_input = SomeStruct; // anything that can be serialized
+let mut stdin = StdIn::default();
+stdin.write(&my_input);
+
+// 5. Run the program
+let output = sdk.execute(exe, vm_config, input)?;
+```
+
+### Using `StdIn`
+
+The `StdIn` struct allows you to format any serializable type into a VM-readable format by passing in a reference to your struct into `StdIn::write` as above. You also have the option to pass in a `&[u8]` into `StdIn::write_bytes`, or a `&[F]` into `StdIn::write_field` where `F` is the `openvm_stark_sdk::p3_baby_bear::BabyBear` field type.
+
+> **Generating CLI Bytes**  
+> To get the VM byte representation of a serializable struct `data` (i.e. for use in the CLI), you can print out the result of `openvm::serde::to_vec(data).unwrap()` in a Rust host program.
+
+## Generating Proofs
+
+After building and transpiling a program, you can then generate a proof. To do so, you need to commit your `VmExe`, generate an `AppProvingKey`, format your input into `StdIn`, and then generate a proof.
+
+```rust
+// 6. Set app configuration
+let app_log_blowup = 2;
+let app_fri_params = FriParameters::standard_with_100_bits_conjectured_security(app_log_blowup);
+let app_config = AppConfig::new(app_fri_params, vm_config);
+
+// 7. Commit the exe
+let app_committed_exe = sdk.commit_app_exe(app_fri_params, exe)?;
+
+// 8. Generate an AppProvingKey
+let app_pk = sdk.app_keygen(app_config)?;
+
+// 9a. Generate a proof
+let proof = sdk.generate_app_proof(app_pk, app_committed_exe, stdin)?;
+// 9b. Generate a proof with an AppProver with custom fields
+let mut app_prover =
+    AppProver::new(app_pk.app_vm_pk.clone(), app_committed_exe)
+        .with_program_name(program_name);
+let proof = app_prover.generate_app_proof(stdin);
+```
+
+## Verifying Proofs
+After generating a proof, you can verify it. To do so, you need your verifying key (which you can get from your `AppProvingKey`) and the output of your `generate_app_proof` call.
+
+```rust
+// 10. Verify your program
+let app_vk = app_pk.get_vk();
+sdk.verify_app_proof(&app_vk, &proof)?;
+```
+
+## End-to-end EVM Proof Generation and Verification
+
+Generating and verifying an EVM proof is an extension of the above process.
+
+```rust
+// 11. Generate the aggregation proving key
+const DEFAULT_PARAMS_DIR: &str = concat!(env!("HOME"), "/.openvm/params/");
+let halo2_params_reader = Halo2ParamsReader::new(DEFAULT_PARAMS_DIR);
+let agg_config = AggConfig::default();
+let agg_pk = sdk.agg_keygen(agg_config, &halo2_params_reader)?;
+
+// 12. Generate an EVM proof
+let proof = sdk.generate_evm_proof(&halo2_params_reader, app_pk, app_committed_exe, agg_pk, stdin)?;
+
+// 13. Generate the SNARK verifier contract
+let verifier = sdk.generate_snark_verifier_contract(&halo2_params_reader, &agg_pk)?;
+
+// 14. Verify the EVM proof
+sdk.verify_evm_proof(&verifier, &proof)?;
+```
+
+Note that `DEFAULT_PARAMS_DIR` is the directory where Halo2 parameters are stored by the `cargo openvm setup` CLI command. For more information on the setup process, see the [onchain verify](../writing-apps/onchain-verify.md) doc.
+
+> ⚠️ **WARNING**  
+> `cargo openvm setup` requires very large amounts of computation and memory (~200 GB).

book/src/advanced-usage/testing-program.md

@@ -10,65 +10,7 @@ printf '\xA0\x86\x01\x00\x00\x00\x00\x00' | cargo run --features std
 
 ### Running with the OpenVM runtime
 
-*TODO*: point to how to install CLI
-
-First to build the guest program:
-```
-cargo axiom build
-```
-
-This compiles the guest program into an [ELF](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) that can be found at `target/riscv32im-risc0-zkvm-elf` directory.
-Next, a host program is needed to run the ELF with openvm runtime. This is where one can configure the openvm with different parameters. There are a few steps:
-
-```rust
-use openvm::transpiler::{openvm_platform::memory::MEM_SIZE, elf::Elf};
-use openvm_circuit::arch::instructions::exe::OpenVmExe
-use openvm_circuit::arch::VmExecutor;
-use openvm_sdk::{config::SdkVmConfig, Sdk, StdIn};
-
-let sdk = Sdk;
-// 1. Build the vm config with the extensions needed.
-// TODO: link to extension
-let vm_config = SdkVmConfig::builder()
-    .system(Default::default())
-    .rv32i(Default::default())
-    .io(Default::default())
-    .build();
-
-// 2. Load the ELF
-let elf = Elf::decode("your_path_to_elf", MEM_SIZE as u32)?;
-let exe = OpenVmExe::from_elf(elf, vm_config.transpiler()).unwrap();
-
-// 3. Prepare the input data
-let my_input = SomeStruct; // anything that can be serialized
-let mut stdin = StdIn::default();
-stdin.write(StdIn::from_bytes(my_input.as_bytes()));
-
-// 4. Run the program
-let executor = VmExecutor::<_, _>::new(vm_config);
-executor.execute(exe, stdin)?;
-```
-Some example host programs can be found [here](https://github.com/openvm-org/openvm/tree/main/benchmarks/src/bin).
-
-### Generating to prove
-
-To generate a proof besides executing the program, instead of using `executor` above (step 4), do the following:
-```rust
-// Some additional configuration.
-let app_log_blowup = 2;
-let app_fri_params = FriParameters::standard_with_100_bits_conjectured_security(app_log_blowup);
-let app_config = AppConfig { ... };
-
-// Keygen and prove
-let app_pk = sdk.app_keygen(app_config)?;
-let app_committed_exe = sdk.commit_app_exe(app_fri_params, exe)?;
-let mut app_prover =
-    AppProver::new(app_pk.app_vm_pk.clone(), app_committed_exe)
-        .with_program_name(program_name);
-let proof = app_prover.generate_app_proof(stdin);
-let app_vk = app_pk.get_vk();
-sdk.verify_app_proof(&app_vk, &proof)?;
-```
+For more information on building, transpiling, running, generating proofs, and verifying proofs with the CLI, see the [CLI](../writing-apps/overview.md)section. To do the same with the SDK, see the [SDK](sdk.md) section.
 
 ## Troubleshooting
 

book/src/writing-apps/compile.md

@@ -7,3 +7,18 @@ First let's define some key terms used in cross-compilation:
 There are multiple things happening in the `cargo openvm build` command as in the section [here](./write-program.md). In short, this command compiles on host to an executable for guest target.
 It first compiles the program normally on your *host* platform with RISC-V and then transpiles it to a different target. See here for some explanation of [cross-compilation](https://rust-lang.github.io/rustup/cross-compilation.html).
 Right now we use `riscv32im-risc0-zkvm-elf` target which is available in the [Rust toolchain](https://doc.rust-lang.org/rustc/platform-support/riscv32im-risc0-zkvm-elf.html), but we will contribute an OpenVM target to Rust in the future.
+
+## Running a Program
+
+After building and transpiling a program, you can execute it using the `run` command. The `run` command has the following arguments:
+
+```bash
+cargo openvm run
+    --exe <path_to_transpiled_program>
+    --config <path_to_app_config>
+    --input <path_to_input>
+```
+
+If `--exe` and/or `--config` are not provided, the command will search for these files in `./openvm/app.vmexe` and `./openvm.toml` respectively. If `./openvm.toml` is not present, a default configuration will be used.
+
+If your program doesn't require inputs, you can (and should) omit the `--input` flag.

book/src/writing-apps/onchain-verify.md

@@ -1 +1,29 @@
 # Onchain Verification
+
+## Generating the Aggregation Proving Key and EVM Verifier Contract
+
+The workflow for generating an end-to-end EVM proof requires first generating an aggregation proving key and EVM verifier contract. This can be done by running the following command:
+
+```bash
+cargo openvm setup
+```
+> ⚠️ **WARNING**  
+> This command requires very large amounts of computation and memory (~200 GB).
+
+Upon a successful run, the command will write `agg.pk` and `verifier.sol` to `~/.openvm/`, where `~` is the directory specified by environment variable `$HOME`. Every command that requires these files will look for them in this directory.
+
+> ⚠️ **WARNING**  
+> If the `$HOME` environment variable is not set, this command may fail.
+
+Note that `cargo openvm setup` may attempt to download other files (i.e. KZG parameters) from an AWS S3 bucket into `~/.openvm/`.
+
+## Generating and Verifying an EVM Proof
+
+To generate and verify an EVM proof, you need to run the following commands:
+
+```bash
+cargo openvm prove evm --input <path_to_input>
+cargo openvm verify evm
+```
+
+These commands are very similar to their `app` subcommand counterparts. For more information on the `prove` and `verify` commands, see the [prove](./prove.md) and [verify](./verify.md) docs.

book/src/writing-apps/overview.md

@@ -1 +1,77 @@
 # Overview of Basic Usage
+
+## Writing a Program
+
+The first step to using OpenVM is to write a Rust program that can be executed by an OpenVM virtual machine. Writing a program for OpenVM is very similar to writing a standard Rust program, with a few key differences necessary to support the OpenVM environment. For more detailed information about writing programs, see the [Writing Programs](./write-program.md) guide.
+
+## Building and Transpiling a Program
+
+At this point, you should have a guest program with a `Cargo.toml` file in the root of your project directory. What's next?
+
+The first thing you will want to do is build and transpile your program using the following command:
+
+```bash
+cargo openvm build
+```
+
+By default this will build the project located in the current directory. To see if it runs correctly, you can try executing it with the following:
+
+```bash
+cargo openvm run --input <path_to_input | hex_string>
+```
+
+Note if your program doesn't require inputs, you can omit the `--input` flag.
+
+For more information on both commands, see the [build](./build.md) docs.
+
+### Inputs
+
+The `--input` field needs to either be a hex string or a file path to a file that will be read as bytes. Note that if your hex string represents a single number, it should be written in little-endian format (as this is what the VM expects). To see how more complex inputs can be converted into a VM-readable format, see the **Using StdIn** section of the [SDK](../advanced-usage/sdk.md) doc.
+
+## Generating a Proof
+
+Given an app configuration TOML file, you first need to generate a proving and verifying key:
+
+```bash
+cargo openvm keygen
+```
+
+After generating the keys, you can generate a proof by running:
+
+```bash
+cargo openvm prove app --input <path_to_input | hex_string>
+```
+
+Again, if your program doesn't require inputs, you can omit the `--input` flag.
+
+For more information on the `keygen` and `prove` commands, see the [prove](./prove.md) doc.
+
+## Verifying a Proof
+
+To verify a proof using the CLI, you need to provide the verifying key and the proof.
+
+```bash
+cargo openvm verify app
+```
+
+For more information on the `verify` command, see the [verify](./verify.md) doc.
+
+## End-to-end EVM Proof Generation and Verification
+
+The process above details the workflow necessary to build, prove, and verify a guest program at the application level. However, to generate the end-to-end EVM proof, you need to (a) setup the aggregation proving key and verifier contract and (b) generate/verify the proof at the EVM level.
+
+To do (a), you need to run the following command. If you've run it previously on your machine, there is no need to do so again. This will write files necessary for EVM proving in `~/.openvm/`.
+
+```bash
+cargo openvm setup
+```
+
+> ⚠️ **WARNING**  
+> This command requires very large amounts of computation and memory (~200 GB).
+
+To do (b), you simply need to replace `app` in `cargo openvm prove` and `cargo openvm verify` as such:
+
+```bash
+cargo openvm prove evm --input <path_to_input | hex_string>
+cargo openvm verify evm
+```

book/src/writing-apps/prove.md

@@ -1 +1,44 @@
 # Generating Proofs
+
+Generating a proof using the CLI is simple - first generate a key, then generate your proof. Using command defaults, this looks like:
+
+```bash
+cargo openvm keygen
+cargo openvm prove [app | evm]
+```
+
+## Key Generation
+
+The `keygen` CLI command has the following optional arguments:
+
+```bash
+cargo openvm keygen
+    --config <path_to_app_config>
+    --output <path_to_app_pk>
+    --vk_output <path_to_app_vk>
+```
+
+If `--config` is not provided, the command will search for `./openvm.toml` and use that as the application configuration if present. If it is not present, a default configuration will be used.
+
+If `--output` and/or `--vk_output` are not provided, the keys will be written to default locations `./openvm/app.pk` and/or `./openvm/app.vk` respectively.
+
+## Proof Generation
+
+The `prove` CLI command has the following optional arguments:
+
+```bash
+cargo openvm prove [app | evm]
+    --app_pk <path_to_app_pk>
+    --exe <path_to_transpiled_program>
+    --input <path_to_input>
+    --output <path_to_output>
+```
+
+If your program doesn't require inputs, you can (and should) omit the `--input` flag.
+
+If `--app_pk` and/or `--exe` are not provided, the command will search for these files in `./openvm/app.pk` and `./openvm/app.vmexe` respectively. Similarly, if `--output` is not provided then the command will write the proof to `./openvm/[app | evm].proof` by default.
+
+The `app` subcommand is used to generate an application-level proof, while the `evm` command generates an end-to-end EVM proof.
+
+> ⚠️ **WARNING**  
+> In order to run the `evm` subcommand, you must have previously called the costly `cargo openvm setup`, which requires very large amounts of computation and memory (~200 GB).