Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add README for integration test structure #2277

Merged
merged 3 commits into from
Aug 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions crates/nargo_cli/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ fn main() {
let test_dir = manifest_dir.join("tests");

generate_execution_success_tests(&mut test_file, &test_dir);
generate_compile_success_tests(&mut test_file, &test_dir);
generate_compile_success_empty_tests(&mut test_file, &test_dir);
generate_compile_failure_tests(&mut test_file, &test_dir);
}

Expand Down Expand Up @@ -81,8 +81,8 @@ fn execution_success_{test_name}() {{
}
}

fn generate_compile_success_tests(test_file: &mut File, test_data_dir: &Path) {
let test_sub_dir = "compile_success";
fn generate_compile_success_empty_tests(test_file: &mut File, test_data_dir: &Path) {
let test_sub_dir = "compile_success_empty";
let test_data_dir = test_data_dir.join(test_sub_dir);

let test_case_dirs =
Expand All @@ -102,14 +102,14 @@ fn generate_compile_success_tests(test_file: &mut File, test_data_dir: &Path) {
test_file,
r#"
#[test]
fn compile_success_{test_name}() {{
fn compile_success_empty_{test_name}() {{
let test_program_dir = PathBuf::from("{test_dir}");

let mut cmd = Command::cargo_bin("nargo").unwrap();
cmd.arg("--program-dir").arg(test_program_dir);
cmd.arg("info");

// `compile_success` tests should be able to compile down to an empty circuit.
// `compile_success_empty` tests should be able to compile down to an empty circuit.
cmd.assert().stdout(predicate::str::contains("| Package")
.and(predicate::str::contains("| Language"))
.and(predicate::str::contains("| ACIR Opcodes | Backend Circuit Size |"))
Expand Down
38 changes: 38 additions & 0 deletions crates/nargo_cli/tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Integration test directory structure

Integration tests for the Noir compiler are broken down into the following directories:

- `compile_failure`: programs which are not valid or unsatisfiable Noir code and so the compiler should reject.
- `compile_success_empty`: programs which are valid satisfiable Noir code but have no opcodes.
- `execution_success`: programs which are valid Noir satisfiable code and have opcodes.

The current testing flow can be thought of as shown:
```mermaid
flowchart TD

subgraph compile_failure
A1[Attempt to compile] --> A2[Assert compilation fails]
end

subgraph compile_success_empty
B1[Attempt to compile] --> B2[Assert compilation succeeds]
B2 --> B3[Assert empty circuit]
end

subgraph execution_success
C1[Attempt to compile] --> C2[Assert compilation succeeds]
C2 --> C3[Write circuit to file]
C3 --> C4[Assert execution succeeds]
C4 --> C5[Write witness to file]

C6[Publish witness + circuit as artifact]
C3 --> C6
C5 --> C6
end
```

## `execution_success` vs `compile_success_empty`

Note that `execution_success` and `compile_success_empty` are distinct as `compile_success_empty` is expected to compile down to an empty circuit. This may not be possible for some argument-less circuits in the situation where instructions have side-effects or certain compiler optimizations are missing, but once moved to `compile_success_empty` a program compiling down to a non-empty circuit is a compiler regression.
kevaundray marked this conversation as resolved.
Show resolved Hide resolved


Empty file.