Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Fuzzer for noir programs #6770

Draft
wants to merge 61 commits into
base: master
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
f098748
something
Rumata888 Nov 21, 2024
1c5d23f
we have a coverage fuzzer
Rumata888 Nov 21, 2024
587fc79
wip
Rumata888 Nov 28, 2024
22508d1
int mutator
Rumata888 Nov 29, 2024
26f9cc2
fast power schedule and dictionary from inputs
Rumata888 Nov 29, 2024
68c1c50
Update mutation count window
Rumata888 Nov 29, 2024
3eff9d7
json
Rumata888 Dec 2, 2024
f32287a
unused cleanup
Rumata888 Dec 2, 2024
917c8cd
listing all
Rumata888 Dec 2, 2024
4348f2e
Remove uint strategy
Rumata888 Dec 4, 2024
41cca17
Fix, better string mutator, acir coverage
Rumata888 Dec 9, 2024
6d06e9d
Useful errors
Rumata888 Dec 9, 2024
b641342
Disk corpus
Rumata888 Dec 11, 2024
b7cea21
remove incorrect prioritization and add splicing
Rumata888 Dec 12, 2024
e98a52d
fix int mutator
Rumata888 Dec 13, 2024
fd2ecb0
Primitive parallelism
Rumata888 Dec 16, 2024
91bf283
relatively fast parallelism
Rumata888 Dec 17, 2024
41c04f3
some cleanup
Rumata888 Dec 23, 2024
368d118
Tracing conditional mov
Rumata888 Dec 23, 2024
2fecd1f
Stupid cmp coverage in brillig
Rumata888 Dec 23, 2024
6813907
Comparison coverage
Rumata888 Dec 23, 2024
cf72aba
refactor start
Rumata888 Dec 23, 2024
3f34dac
fixed parallelism
Rumata888 Dec 26, 2024
1bf3d63
connected show output
Rumata888 Jan 1, 2025
6a48a2e
autoejection from corpus and better update times
Rumata888 Jan 3, 2025
588a680
remove noisy print
Rumata888 Jan 3, 2025
1fe9b6e
Separate asserts
Rumata888 Jan 3, 2025
ed4e87b
storing failing inputs
Rumata888 Jan 3, 2025
8cbcbd0
started refactoring
Rumata888 Jan 6, 2025
1f5b788
Merge branch 'master' into my domain
Rumata888 Jan 6, 2025
a31afef
fixes
Rumata888 Jan 6, 2025
0613f61
some refactoring
Rumata888 Jan 6, 2025
aa5e26f
refactoring, continued
Rumata888 Jan 6, 2025
9f21a45
refactored field mutations
Rumata888 Jan 7, 2025
4083782
feat: don't simplify SSA instructions when creating them from a strin…
asterite Jan 6, 2025
f98b966
feat: lock on Nargo.toml on several nargo commands (#6941)
asterite Jan 6, 2025
07457c0
feat!: turn CannotReexportItemWithLessVisibility into an error (#6952)
asterite Jan 6, 2025
38c89f4
chore: Separate unconstrained functions during monomorphization (#6894)
jfecher Jan 6, 2025
b728151
chore: Move comment as part of #6945 (#6959)
vezenovm Jan 6, 2025
6c60f92
feat(ssa): Immediately simplify away RefCount instructions in ACIR fu…
aakoshh Jan 6, 2025
8e870ed
chore: simplify boolean in a mul of a mul (#6951)
guipublic Jan 6, 2025
368356f
feat: don't report warnings for dependencies (#6926)
asterite Jan 6, 2025
432c477
fix: error on missing function parameters (#6967)
asterite Jan 7, 2025
ca46630
fix: don't fail parsing macro if there are parser warnings (#6969)
asterite Jan 7, 2025
9ceb9f9
feat!: turn TypeIsMorePrivateThenItem into an error (#6953)
asterite Jan 7, 2025
e9c0fed
fix: Start RC at 1 again (#6958)
jfecher Jan 7, 2025
df4b035
fix: wrong module to lookup trait when using crate or super (#6974)
asterite Jan 7, 2025
1c19ac2
feat!: update `aes128_encrypt` to return an array (#6973)
TomAFrench Jan 7, 2025
9cca916
feat: `--pedantic-solving` flag (#6716)
michaeljklein Jan 7, 2025
30514fe
feat!: type-check trait default methods (#6645)
asterite Jan 7, 2025
8879eee
feat!: require trait method calls (`foo.bar()`) to have the trait in …
asterite Jan 8, 2025
6d85cd6
fix: do not panic on indices which are not valid `u32`s (#6976)
TomAFrench Jan 8, 2025
6078d5c
docs and race fix
Rumata888 Jan 8, 2025
f9a033b
String mutation refactor
Rumata888 Jan 9, 2025
bdf5b57
a bit more of field and string refactor + added swap to string mutators
Rumata888 Jan 13, 2025
dc0a0b4
Int mutator refactor
Rumata888 Jan 13, 2025
e41f68f
comments
Rumata888 Jan 15, 2025
306f2a6
Merge branch 'master' into my domain
Rumata888 Jan 15, 2025
a82f65f
separate out the dictionary
Rumata888 Jan 16, 2025
2bb2f31
renamed folder
Rumata888 Jan 16, 2025
f610496
moved configs and started working on array splicing
Rumata888 Jan 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Separate asserts
  • Loading branch information
Rumata888 committed Jan 3, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 1fe9b6e36d45604d756fb4311a40f95dc048e65b
7 changes: 6 additions & 1 deletion acvm-repo/acvm/src/pwg/mod.rs
Original file line number Diff line number Diff line change
@@ -561,7 +561,12 @@ impl<'a, F: AcirField, B: BlackBoxFunctionSolver<F>> ACVM<'a, F, B> {
)?,
};

let result = solver.solve().map_err(|err| self.map_brillig_error(err))?;
let result = solver.solve().map_err(|err| {
if self.brillig_fuzzing_active {
self.brillig_fuzzing_trace = Some((&solver).get_fuzzing_trace())
};
self.map_brillig_error(err)
})?;

match result {
BrilligSolverStatus::ForeignCallWait(foreign_call) => {
2 changes: 1 addition & 1 deletion compiler/noirc_frontend/src/ast/function.rs
Original file line number Diff line number Diff line change
@@ -113,7 +113,7 @@ impl From<FunctionDefinition> for NoirFunction {
Some(FunctionAttribute::Builtin(_)) => FunctionKind::Builtin,
Some(FunctionAttribute::Foreign(_)) => FunctionKind::LowLevel,
Some(FunctionAttribute::Test { .. }) => FunctionKind::Normal,
Some(FunctionAttribute::FuzzingHarness) => FunctionKind::Normal,
Some(FunctionAttribute::FuzzingHarness { .. }) => FunctionKind::Normal,
Some(FunctionAttribute::Oracle(_)) => FunctionKind::Oracle,
Some(FunctionAttribute::Recursive) => FunctionKind::Recursive,
Some(FunctionAttribute::Fold) => FunctionKind::Normal,
30 changes: 25 additions & 5 deletions compiler/noirc_frontend/src/hir/def_map/mod.rs
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ use crate::hir::Context;
use crate::node_interner::{FuncId, GlobalId, NodeInterner, StructId};
use crate::parse_program;
use crate::parser::{ParsedModule, ParserError};
use crate::token::{FunctionAttribute, SecondaryAttribute, TestScope};
use crate::token::{FunctionAttribute, FuzzingScope, SecondaryAttribute, TestScope};
use fm::{FileId, FileManager};
use noirc_arena::{Arena, Index};
use noirc_errors::Location;
@@ -194,9 +194,9 @@ impl CrateDefMap {
if let Some(func_id) = id.as_function() {
let attributes = interner.function_attributes(&func_id);
match attributes.function() {
Some(FunctionAttribute::FuzzingHarness) => {
Some(FunctionAttribute::FuzzingHarness(scope)) => {
let location = interner.function_meta(&func_id).name.location;
Some(FuzzingHarness::new(func_id, location))
Some(FuzzingHarness::new(func_id, scope.clone(), location))
}
_ => None,
}
@@ -406,12 +406,13 @@ impl TestFunction {

pub struct FuzzingHarness {
id: FuncId,
scope: FuzzingScope,
location: Location,
}

impl FuzzingHarness {
fn new(id: FuncId, location: Location) -> Self {
FuzzingHarness { id, location }
fn new(id: FuncId, scope: FuzzingScope, location: Location) -> Self {
FuzzingHarness { id, scope, location }
}

/// Returns the function id of the test function
@@ -422,4 +423,23 @@ impl FuzzingHarness {
pub fn file_id(&self) -> FileId {
self.location.file
}

/// Returns true if the fuzzing harness has been specified to fail only under specific reason
/// This is done by annotating the function with
/// `#[fuzz(only_fail_with = "reason")]`
pub fn only_fail_enabled(&self) -> bool {
match self.scope {
FuzzingScope::OnlyFailWith { .. } => true,
FuzzingScope::None => false,
}
}

/// Returns the reason for the fuzzing harness to fail if specified
/// by the user.
pub fn failure_reason(&self) -> Option<String> {
match &self.scope {
FuzzingScope::None => None,
FuzzingScope::OnlyFailWith { reason } => Some(reason.clone()),
}
}
}
8 changes: 8 additions & 0 deletions compiler/noirc_frontend/src/lexer/errors.rs
Original file line number Diff line number Diff line change
@@ -22,6 +22,8 @@ pub enum LexerErrorKind {
MalformedFuncAttribute { span: Span, found: String },
#[error("Malformed test attribute")]
MalformedTestAttribute { span: Span },
#[error("Malformed fuzz attribute")]
MalformedFuzzAttribute { span: Span },
#[error("{:?} is not a valid inner attribute", found)]
InvalidInnerAttribute { span: Span, found: String },
#[error("Logical and used instead of bitwise and")]
@@ -64,6 +66,7 @@ impl LexerErrorKind {
LexerErrorKind::IntegerLiteralTooLarge { span, .. } => *span,
LexerErrorKind::MalformedFuncAttribute { span, .. } => *span,
LexerErrorKind::MalformedTestAttribute { span, .. } => *span,
LexerErrorKind::MalformedFuzzAttribute { span, .. } => *span,
LexerErrorKind::InvalidInnerAttribute { span, .. } => *span,
LexerErrorKind::LogicalAnd { span } => *span,
LexerErrorKind::UnterminatedBlockComment { span } => *span,
@@ -117,6 +120,11 @@ impl LexerErrorKind {
"The test attribute can be written in one of these forms: `#[test]`, `#[test(should_fail)]` or `#[test(should_fail_with = \"message\")]`".to_string(),
*span,
),
LexerErrorKind::MalformedFuzzAttribute { span } => (
"Malformed fuzz attribute".to_string(),
"The fuzz attribute can be written in one of these forms: `#[fuzz]` or `#[fuzz(only_fail_with = \"message\")]`".to_string(),
*span,
),
LexerErrorKind::InvalidInnerAttribute { span, found } => (
"Invalid inner attribute".to_string(),
format!(" {found} is not a valid inner attribute"),
28 changes: 24 additions & 4 deletions compiler/noirc_frontend/src/lexer/token.rs
Original file line number Diff line number Diff line change
@@ -647,6 +647,26 @@ impl fmt::Display for TestScope {
}
}

/// FuzzingScopr is used to specify additional annotations for fuzzing harnesses
#[derive(PartialEq, Eq, Hash, Debug, Clone, PartialOrd, Ord)]
pub enum FuzzingScope {
/// If a fuzzing harness has a scope of OnlyFailWith, then it will only detect an assert
/// if it fails with the specified reason.
OnlyFailWith {
reason: String,
},
None,
}

impl fmt::Display for FuzzingScope {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
FuzzingScope::None => write!(f, ""),
FuzzingScope::OnlyFailWith { reason } => write!(f, "(only_fail_with = {reason:?})"),
}
}
}

#[derive(PartialEq, Eq, Debug, Clone)]
// Attributes are special language markers in the target language
// An example of one is `#[SHA256]` . Currently only Foreign attributes are supported
@@ -687,7 +707,7 @@ impl Attributes {
}

pub fn is_fuzzing_harness(&self) -> bool {
matches!(self.function(), Some(FunctionAttribute::FuzzingHarness))
matches!(self.function(), Some(FunctionAttribute::FuzzingHarness(_)))
}

/// True if these attributes mean the given function is an entry point function if it was
@@ -763,7 +783,7 @@ pub enum FunctionAttribute {
Fold,
NoPredicates,
InlineAlways,
FuzzingHarness,
FuzzingHarness(FuzzingScope),
}

impl FunctionAttribute {
@@ -828,7 +848,7 @@ impl FunctionAttribute {
FunctionAttribute::Fold => "fold",
FunctionAttribute::NoPredicates => "no_predicates",
FunctionAttribute::InlineAlways => "inline_always",
FunctionAttribute::FuzzingHarness => "fuzz",
FunctionAttribute::FuzzingHarness(_) => "fuzz",
}
}
}
@@ -844,7 +864,7 @@ impl fmt::Display for FunctionAttribute {
FunctionAttribute::Fold => write!(f, "#[fold]"),
FunctionAttribute::NoPredicates => write!(f, "#[no_predicates]"),
FunctionAttribute::InlineAlways => write!(f, "#[inline_always]"),
FunctionAttribute::FuzzingHarness => write!(f, "#[fuzz]"),
FunctionAttribute::FuzzingHarness(scope) => write!(f, "#[fuzz{scope}]"),
}
}
}
55 changes: 47 additions & 8 deletions compiler/noirc_frontend/src/parser/parser/attributes.rs
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ use crate::ast::{Expression, ExpressionKind, Ident, Literal, Path};
use crate::lexer::errors::LexerErrorKind;
use crate::parser::labels::ParsingRuleLabel;
use crate::parser::ParserErrorReason;
use crate::token::{Attribute, FunctionAttribute, MetaAttribute, TestScope, Token};
use crate::token::{Attribute, FunctionAttribute, FuzzingScope, MetaAttribute, TestScope, Token};
use crate::token::{CustomAttribute, SecondaryAttribute};

use super::parse_many::without_separator;
@@ -54,6 +54,8 @@ impl<'a> Parser<'a> {
/// | 'test'
/// | 'test' '(' 'should_fail' ')'
/// | 'test' '(' 'should_fail_with' '=' string ')'
/// | 'fuzz'
/// | 'fuzz' '(' 'only_fail_with' '=' string ')'
///
/// SecondaryAttribute
/// = 'abi' '(' AttributeValue ')'
@@ -141,12 +143,16 @@ impl<'a> Parser<'a> {
} else if let Some(path) = self.parse_path_no_turbofish() {
if let Some(ident) = path.as_ident() {
if ident.0.contents == "test" {
// The test attribute is the only secondary attribute that has `a = b` in its syntax
// The test attribute is a secondary attribute that has `a = b` in its syntax
// (`should_fail_with = "..."``) so we parse it differently.
self.parse_test_attribute(start_span)
} else if ident.0.contents == "fuzz" {
// The fuzz attribute is a secondary attribute that has `a = b` in its syntax
// (`only_fail_with = "..."``) so we parse it differently.
self.parse_fuzz_attribute(start_span)
} else {
// Every other attribute has the form `name(arg1, arg2, .., argN)`
self.parse_ident_attribute_other_than_test(ident, start_span)
self.parse_ident_attribute_other_than_test_and_fuzz(ident, start_span)
}
} else {
// This is a Meta attribute with the syntax `path(arg1, arg2, .., argN)`
@@ -168,7 +174,7 @@ impl<'a> Parser<'a> {
}))
}

fn parse_ident_attribute_other_than_test(
fn parse_ident_attribute_other_than_test_and_fuzz(
&mut self,
ident: &Ident,
start_span: Span,
@@ -204,10 +210,6 @@ impl<'a> Parser<'a> {
"foreign" => self.parse_single_name_attribute(ident, arguments, start_span, |name| {
Attribute::Function(FunctionAttribute::Foreign(name))
}),
"fuzz" => {
let attr = Attribute::Function(FunctionAttribute::FuzzingHarness);
self.parse_no_args_attribute(ident, arguments, attr)
}
"inline_always" => {
let attr = Attribute::Function(FunctionAttribute::InlineAlways);
self.parse_no_args_attribute(ident, arguments, attr)
@@ -311,6 +313,43 @@ impl<'a> Parser<'a> {
Attribute::Function(FunctionAttribute::Test(scope))
}

fn parse_fuzz_attribute(&mut self, start_span: Span) -> Attribute {
let scope = if self.eat_left_paren() {
let scope = if let Some(ident) = self.eat_ident() {
match ident.0.contents.as_str() {
"only_fail_with" => {
self.eat_or_error(Token::Assign);
if let Some(reason) = self.eat_str() {
Some(FuzzingScope::OnlyFailWith { reason: reason })
} else {
None
}
}
_ => None,
}
} else {
None
};
self.eat_or_error(Token::RightParen);
scope
} else {
Some(FuzzingScope::None)
};

self.skip_until_right_bracket();

let scope = if let Some(scope) = scope {
scope
} else {
self.errors.push(
LexerErrorKind::MalformedFuzzAttribute { span: self.span_since(start_span) }.into(),
);
FuzzingScope::None
};

Attribute::Function(FunctionAttribute::FuzzingHarness(scope))
}

fn parse_single_name_attribute<F>(
&mut self,
ident: &Ident,
4 changes: 2 additions & 2 deletions tooling/greybox_fuzzer/src/coverage.rs
Original file line number Diff line number Diff line change
@@ -102,14 +102,14 @@ pub struct SingleTestCaseCoverage {
impl SingleTestCaseCoverage {
pub fn new(
testcase_id: TestCaseId,
acir_witnesses: &WitnessStack<FieldElement>,
acir_witness_stack: &Option<WitnessStack<FieldElement>>,
brillig_coverage: Vec<u32>,
potential_bool_witness_list: &PotentialBoolWitnessList,
) -> Self {
// Process all booleans
let mut acir_bool_coverage = Vec::new();
// If the witness stack was not empty
if acir_witnesses.length() == 1 {
if let Some(acir_witnesses) = acir_witness_stack {
let witness_map = &acir_witnesses.peek().unwrap().witness;

for potential_bool_witness_index in potential_bool_witness_list.witness.iter() {
Loading