chore: update project layout (#1086)

* chore: update project layout

* chore: readme

* fix: link

crates/toolchain/build/Cargo.toml

@@ -5,7 +5,7 @@ authors.workspace = true
 edition.workspace = true
 homepage.workspace = true
 repository.workspace = true
-description = "OpenVM build tool"
+description = "OpenVM build tools"
 
 [dependencies]
 openvm-platform = { workspace = true }

crates/toolchain/openvm/README.md

@@ -0,0 +1,3 @@
+# OpenVM Standard Library
+
+The OpenVM standard library for use in Rust guest programs.

crates/vm/README.md

@@ -1,3 +1 @@
-# STARK Virtual Machine
-
-This crate contains STARK implementations for building minimal Virtual Machines. The ISA spec is [here](https://hackmd.io/i-EIS7KvSHeRLOnL6vdphA).
+# OpenVM Circuit Framework

docs/crates/benchmarks.md

@@ -1,5 +1,16 @@
 # Benchmarks
 
+### Latest Benchmark Results
+
+Latest benchmark results can be found [here](https://github.com/openvm-org/openvm/blob/benchmark-results/index.md).
+These are run via [github workflows](./.github/workflows/benchmark-call.yml) and should always be up to date with the latest `main` branch.
+
+### Adding a Benchmark
+
+See [How to Add a Benchmark](../../benchmarks/README.md#how-to-add-a-benchmark).
+
+### Running Benchmarks Locally
+
 To run benchmarks, install python3 and run (from root of repo):
 
 ```bash
@@ -18,7 +29,9 @@ Currently the processing is done automatically at the end of `bench.py`. The scr
 
 ### Flamegraphs
 
-Flamegraphs to visualize the metrics collected by the VM cycle tracker can be generated if you have [inferno-flamegraph](https://crates.io/crates/inferno) installed. Install via
+While traditional flamegraphs generated from instrumenting a proving binary run on the host machine are useful,
+for more detailed profiling we generate special flamegraphs that visualize VM-specific metrics such as cycle counts and trace cell usage with stack traces.
+These flamegraphs can be generated if you have [inferno-flamegraph](https://crates.io/crates/inferno) installed. Install via
 
 ```bash
 cargo install inferno
@@ -31,8 +44,3 @@ python ci/scripts/metric_unify/flamegraph.py <path to json>
 ```
 
 The flamegraphs will be written to `*.svg` files in `.bench_metrics/flamegraphs` with respect to the repo root.
-
-### Latest Benchmark Results
-
-Latest benchmark results can be found [here](https://github.com/openvm-org/openvm/blob/benchmark-results/index.md).
-These are run via [github workflows](./.github/workflows/benchmark-call.yml) and should always be up to date with the latest `main` branch.

docs/repo/layout.md

@@ -7,23 +7,22 @@ The main components of the repository are:
   - [Benchmarks](#benchmarks)
   - [CI](#ci)
   - [CLI](#cli)
-  - [VM SDK](#vm-sdk)
-  - [Rust Toolchain](#rust-toolchain)
-  - [VM Framework](#vm-framework)
+  - [SDK](#sdk)
+  - [Toolchain](#toolchain)
+  - [Circuit Framework](#circuit-framework)
   - [Circuit Foundations](#circuit-foundations)
-  - [Proof System](#proof-system)
   - [Extensions](#extensions)
     - [RV32IM](#rv32im)
     - [Native Recursion](#native-recursion)
     - [Keccak256](#keccak256)
     - [Big Integers](#big-integers)
-    - [Modular Arithmetic](#modular-arithmetic)
+    - [Algebra (Modular Arithmetic)](#algebra-modular-arithmetic)
     - [Elliptic Curve Cryptography](#elliptic-curve-cryptography)
-    - [Pairing](#pairing)
+    - [Elliptic Curve Pairing](#elliptic-curve-pairing)
 
 ### Documentation
 
-Contributor documentation is in [`docs`](../../docs) and end-user documentation is in [`book`](../../book).
+Contributor documentation is in [`docs`](../../docs) and user documentation is in [`book`](../../book).
 
 ### Benchmarks
 
@@ -37,81 +36,78 @@ Scripts for CI use and metrics post-processing are in [`ci`](../../ci).
 
 Command-line binary to compile, execute, and prove guest programs is in [`cli`](../../crates/cli).
 
-### VM SDK
+### SDK
 
-- [`sdk`](../../crates/sdk): The developer SDK for the VM. It includes the OpenVM aggregation programs to support continuations for all VMs in the framework, and well as local aggregation scheduling implementation. It provides the final interface for proving an arbitrary program for a target VM. Includes utilities to generate final onchain SNARK verifier contract.
+- [`sdk`](../../crates/sdk): The developer SDK for the VM. It includes the OpenVM aggregation programs to support continuations for all VMs in the framework, and well as a local aggregation scheduling implementation. It provides the final interface for proving an arbitrary program for a target VM. The SDK includes functionality to generate the final onchain SNARK verifier contract.
 
-### Rust Toolchain
+### Toolchain
 
 - [`openvm`](../../crates/toolchain/openvm): The OpenVM standard library to be imported by guest programs. Contains `main` function entrypoint and standard intrinsic functions for IO.
-- [`openvm-platform`](../../crates/toolchain/platform): Rust runtime for RV32IM target using OpenVM intrinsic for system termination.
-- [`openvm-transpiler`](../../crates/toolchain/transpiler): Transpiler for converting RISC-V ELF with custom instructions into OpenVM executable with OpenVM instructions.
+- [`openvm-platform`](../../crates/toolchain/platform): Rust runtime for RV32IM target using OpenVM intrinsic for system termination. This crate is re-exported by the `openvm` crate.
+- [`openvm-build`](../../crates/toolchain/build): Library of build tools for compiling Rust to the RISC-V target, built on top of `cargo`.
+- [`openvm-transpiler`](../../crates/toolchain/transpiler): Transpiler for converting RISC-V ELF with custom instructions into OpenVM executable with OpenVM instructions. This crate contains the `TranspilerExtension` trait and a `Transpiler` struct which supports adding custom `TranspilerExtension` implementations.
+- [`openvm-instructions`](../../crates/toolchain/instructions): OpenVM instruction struct and trait definitions. Also includes some system instruction definitions.
+- [`openvm-instructions-derive`](../../crates/toolchain/instructions/derive): Procedural macros to derive traits for OpenVM instructions.
 - [`openvm-macros-common`](../../crates/toolchain/macros): Common library for parsing utilities shared across procedural macros used for custom instruction setup in guest programs.
-- [`openvm-toolchain-tests`](../../crates/toolchain/tests): Testing of Rust toolchain including all official RISC-V 32-bit IM test vectors.
+- [`openvm-toolchain-tests`](../../crates/toolchain/tests): Testing of Rust toolchain including all official RISC-V 32-bit IM test vectors. Currently this is a monolithic crate with tests across many different extensions. We will soon refactor the tests to be more modular.
 
-### VM Framework
+### Circuit Framework
 
 - [`openvm-circuit`](../../crates/vm): The VM circuit framework. It includes the struct and trait definitions used throughout the architecture, as well as the system chips.
 - [`openvm-circuit-derive`](../../crates/vm/derive): Procedural macros to derive traits in the VM circuit framework.
-- [`openvm-instructions`](../../crates/toolchain/instructions): OpenVM instruction struct and trait definitions.
-- [`openvm-instructions-derive`](../../crates/toolchain/instructions/derive): Procedural macros to derive traits for OpenVM instructions.
 
 ### Circuit Foundations
 
 - [`openvm-circuit-primitives`](../../crates/circuits/primitives): Primitive chips and sub-chips for standalone use in any circuit.
-- [`openvm-circuit-primitives-derive`](../../crates/circuits/derive): Procedural macros for use in circuit to derive traits.
-- [`openvm-poseidon2-air`](../../crates/circuits/poseidon2-air): Standalone poseidon2 AIR implementation.
-- [`openvm-mod-circuit-builder`](../../crates/circuits/mod-builder): General builder for generating chip for any modular arithmetic expression for a compile-time modulus.
-
-### Proof System
-
-- [`openvm-stark-backend`](../../crates/stark-backend): General purpose STARK proving system with multi-trace and logup support, built on top of plonky3.
-- [`openvm-stark-sdk`](../../crates/stark-sdk): Low-level SDK for use with STARK backend to generate proofs for specific STARK configurations.
+- [`openvm-circuit-primitives-derive`](../../crates/circuits/primitives/derive): Procedural macros for use in circuit to derive traits.
+- [`openvm-poseidon2-air`](../../crates/circuits/poseidon2-air): Standalone poseidon2 AIR implementation which is configurable based on the desired maximum constraint degree.
+- [`openvm-mod-circuit-builder`](../../crates/circuits/mod-builder): General builder for generating a chip for any modular arithmetic expression for a modulus known at compile time.
 
 ### Extensions
 
 The toolchain, ISA, and VM are simultaenously extendable. All non-system functionality is implemented via extensions, which may be moved to standalone repositories in the future but are presently in this repository for maintainer convenience.
 
 #### RV32IM
 
-- [`openvm-rv32im-circuit`](../../extensions/rv32im/circuit): VM circuit extension for RV32IM instructions, including IO operations.
-- [`openvm-rv32im-transpiler`](../../extensions/rv32im/transpiler): Transpiler extension for RV32IM instructions.
-- [`openvm-rv32im-guest`](../../extensions/rv32im/guest): Guest library for RV32IM instructions.
+- [`openvm-rv32im-circuit`](../../extensions/rv32im/circuit): Circuit extension for RV32IM instructions and IO instructions.
+- [`openvm-rv32im-transpiler`](../../extensions/rv32im/transpiler): Transpiler extension for RV32IM instructions and IO instructions.
+- [`openvm-rv32im-guest`](../../extensions/rv32im/guest): Guest library for RV32IM instructions and IO instructions. This is re-exported by the `openvm` crate for convenience.
 
 #### Native Recursion
 
-- [`openvm-native-circuit`](../../extensions/native/circuit/): VM circuit extension for native instructions operating on field elements.
+- [`openvm-native-circuit`](../../extensions/native/circuit/): Circuit extension for native instructions operating on field elements.
 - [`openvm-native-compiler`](../../extensions/native/compiler/): Implementation of compiler from a Rust embedded DSL to OpenVM assembly targeting the native kernel extension. The eDSL also has a static mode to support compilation to a Halo2 circuit.
-- [`openvm-native-recursion`](../../extensions/native/recursion): Library written in the native eDSL with functions to verify arbitrary STARK proofs. Library supports compilation to Halo2 circuit.
+- [`openvm-native-recursion`](../../extensions/native/recursion): Library written in the native eDSL with functions to verify arbitrary STARK proofs. The library also supports compilation to a Halo2 circuit.
 
 #### Keccak256
 
-- [`openvm-keccak256-circuit`](../../extensions/keccak256/circuit): VM circuit extension for `keccak256` hash function.
-- [`openvm-keccak256-transpiler`](../../extensions/keccak256/transpiler): Transpiler extension for `keccak256` hash function.
-- [`openvm-keccak256-guest`](../../extensions/keccak256/guest): Guest library with intrinsic function for `keccak256` hash function.
+- [`openvm-keccak256-circuit`](../../extensions/keccak256/circuit): Circuit extension for the `keccak256` hash function.
+- [`openvm-keccak256-transpiler`](../../extensions/keccak256/transpiler): Transpiler extension for the `keccak256` hash function.
+- [`openvm-keccak256-guest`](../../extensions/keccak256/guest): Guest library with intrinsic function for the `keccak256` hash function.
 
 #### Big Integers
 
-- [`openvm-bigint-circuit`](../../extensions/bigint/circuit): VM circuit extension for `I256` and `U256` big integer operations.
+- [`openvm-bigint-circuit`](../../extensions/bigint/circuit): Circuit extension for `I256` and `U256` big integer operations.
 - [`openvm-bigint-transpiler`](../../extensions/bigint/transpiler): Transpiler extension for `I256` and `U256` big integer operations.
 - [`openvm-bigint-guest`](../../extensions/bigint/guest): Guest library with `I256` and `U256` big integers operations using intrinsics for underlying operations.
 
-#### Modular Arithmetic
+#### Algebra (Modular Arithmetic)
 
-- [`openvm-algebra-circuit`](../../extensions/algebra/circuit): VM circuit extension for modular arithmetic for arbitrary compile-time modulus. Supports modular arithmetic and Fp2 operations.
-- [`openvm-algebra-transpiler`](../../extensions/algebra/transpiler): Transpiler extension for modular arithmetic for arbitrary compile-time modulus. Supports modular arithmetic and Fp2 operations.
-- [`openvm-algebra-guest`](../../extensions/algebra/guest): Guest library with traits for modular arithmetic and Fp2 operations.
+- [`openvm-algebra-circuit`](../../extensions/algebra/circuit): Circuit extension for modular arithmetic for arbitrary compile-time modulus. Supports modular arithmetic and complex field extension operations.
+- [`openvm-algebra-transpiler`](../../extensions/algebra/transpiler): Transpiler extension for modular arithmetic for arbitrary compile-time modulus. Supports modular arithmetic and complex field extension operations.
+- [`openvm-algebra-guest`](../../extensions/algebra/guest): Guest library with traits for modular arithmetic and complex field extension operations.
 - [`openvm-algebra-moduli-setup`](../../extensions/algebra/moduli-setup): Procedural macros for use in guest program to generate modular arithmetic struct with custom intrinsics for compile-time modulus.
+- [`openvm-algebra-complex-macros`](../../extensions/algebra/guest/src/field/complex-macros): Procedural macros for use in guest program to generate complex field struct with custom intrinsics for compile-time modulus.
 
 #### Elliptic Curve Cryptography
 
-- [`openvm-ecc-circuit`](../../extensions/ecc/circuit): VM circuit extension for Weierstrass elliptic curve operations for arbitrary compile-time curve.
+- [`openvm-ecc-circuit`](../../extensions/ecc/circuit): Circuit extension for Weierstrass elliptic curve operations for arbitrary compile-time curve.
 - [`openvm-ecc-transpiler`](../../extensions/ecc/transpiler): Transpiler extension for Weierstrass elliptic curve operations for arbitrary compile-time curve.
-- [`openvm-ecc-guest`](../../extensions/ecc/guest): Guest library with elliptic curve constants for Secp256k1 and functions using custom intrinsics, including ECDSA.
+- [`openvm-ecc-guest`](../../extensions/ecc/guest): Guest library with traits for elliptic curve cryptography. Includes implementations of ECDSA and multi-scalar multiplication.
 - [`openvm-ecc-sw-setup`](../../extensions/ecc/sw-setup): Procedural macros for use in guest program to generate short Weierstrass curve struct with custom intrinsics for compile-time curve.
 
-#### Pairing
+#### Elliptic Curve Pairing
 
-- [`openvm-pairing-circuit`](../../extensions/pairing/circuit): VM circuit extension for optimal Ate pairing on arbitrary compile-time elliptic curves, including BN254 and BLS12-381.
-- [`openvm-pairing-transpiler`](../../extensions/pairing/transpiler): Transpiler extension for optimal Ate pairing on arbitrary compile-time elliptic curves, including BN254 and BLS12-381.
-- [`openvm-pairing-guest`](../../extensions/pairing/guest): Guest library with optimal Ate pairing on elliptic curves, including BN254 and BLS12-381 and associated constants. Also includes elliptic curve operations for VM runtime with the `halo2curves` feature gate.
+- [`openvm-pairing-circuit`](../../extensions/pairing/circuit): Circuit extension for optimal Ate pairing on BN254 and BLS12-381 curves.
+- [`openvm-pairing-transpiler`](../../extensions/pairing/transpiler): Transpiler extension for optimal Ate pairing on BN254 and BLS12-381.
+- [`openvm-pairing-guest`](../../extensions/pairing/guest): Guest library with optimal Ate pairing on BN254 and BLS12-381 and associated constants. Also includes elliptic curve operations for VM runtime with the `halo2curves` feature gate.

docs/specs/README.md

@@ -20,6 +20,19 @@ The framework is designed to be extendable via external crates _without forking_
 VM extensions provide a way to simultaneously extend the VM with new chips, opcodes, and toolchain support for these opcodes.
 A new extension of the overall architecture consists of three components:
 
-- Library: the guest library that compiles program code (usually in Rust) into RISC-V assembly with custom instructions.
+- Guest library: the guest library that compiles program code (usually in Rust) into RISC-V assembly with custom instructions.
 - Transpiler extension: extend the transpiler to specify how newly introduced custom RISC-V instructions should be transpiled into custom OpenVM instructions.
-- VM extension: define new chips and assign them to handle the new opcodes.
+- Circuit extension: define new chips and assign them to handle the new opcodes.
+
+These three components should be organized into three separate crates. When introducing a new extension with name `$name`, we recommend naming the crates as follows:
+
+- `openvm-$name-guest`: the guest library crate. This crate specifies the custom RISC-V instructions to be added. To avoid opcode collisions, we keep a list of currently supported custom instructions in [this](./RISCV.md) file.
+- `openvm-$name-transpiler`: the transpiler extension crate. This crate needs to import `openvm-$name-guest` to get the custom RISC-V instruction definitions. The `openvm-$name-transpiler` crate specifies the new OpenVM instruction definitions (represented in field elements) as well as the transpiler extension.
+- `openvm-$name-circuit`: the circuit extension crate that defines new chips. This crate needs to import `openvm-$name-transpiler` to get the new OpenVM instruction definitions.
+
+## Specifications
+
+- [Circuit Architecture](./circuit.md)
+- [Instruction Set Architecture](./ISA.md)
+- [RISC-V custom instructions and transpiler](./RISCV.md)
+- [Continuations](./continuations.md)