Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] More Advanced Transaction Filtering #389

Merged
merged 7 commits into from
Jun 11, 2024

Conversation

CapCap
Copy link
Contributor

@CapCap CapCap commented May 25, 2024

an MVP version of the transaction filtering crate

Performance

On a random 3mb proto of 1000txns during taptos, running the "everything" filter (looped) gives:

BENCH: Took 13.225167ms for 1000000 transactions (13ns each)

Random 10mb proto from April 25th with "everything" filter:

BENCH: Took 21.815375ms for 1000000 transactions (21ns each)

Graffio 100mb proto ("everything" filter, but with or):

BENCH: Took 31.559333ms for 1000000 transactions (31ns each)

@CapCap CapCap force-pushed the transaction_filters branch 2 times, most recently from cb2d072 to 30ec90f Compare May 25, 2024 04:49
@CapCap CapCap changed the title Transaction Filtering [WIP] Transaction Filtering May 25, 2024
@CapCap CapCap force-pushed the transaction_filters branch 4 times, most recently from 52860cd to 282205f Compare May 31, 2024 02:21
Copy link
Collaborator

@banool banool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial feedback, I didn't dive too deep into the specific filters.

rust/transaction-filter/src/filter_operator.rs Outdated Show resolved Hide resolved
WriteSetChangeFilter(WriteSetChangeFilter),
}

impl Filterable<Transaction> for APIFilter {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use enum_dispatch / ambassador to help eliminate some of the boilerplate here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enum dispatchhhhh

rust/transaction-filter/src/traits.rs Show resolved Hide resolved
pub trait Filterable<T> {
/// Whether this filter is correctly configured/initialized
/// Any call to `is_valid` is responsible for recursively checking the validity of any nested filters
fn is_valid(&self) -> Result<(), Error>;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I imagine the reason we have is_valid is because we can't do it in constructors because these structs might get built directly from configuration, and we can't do it in the parsing logic because serde doesn't provide powerful enough attributes to let us do validity checking at parsing time, yeah?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the constructors of what?

My assumption for these filters is they will be around for a long time, and different refs will flow through them- there's only ever one copy of these

Injecting it into the serialization would be on the GRPC side, which would be inside of poem, and I wouldn't want to mess with that 😬

rust/transaction-filter/src/filters/event_filter.rs Outdated Show resolved Hide resolved
rust/transaction-filter/src/filters/event_filter.rs Outdated Show resolved Hide resolved
rust/transaction-filter/src/filters/move_module.rs Outdated Show resolved Hide resolved
}
}

fn is_allowed(&self, txn: &Transaction) -> bool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't wanna nerd snipe you but it'd be neat if this function at the top level told you which filter allowed it if this is true 😛

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like if a transaction got filtered out, it'd be cool to see which filter exactly it failed on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sigh :-p

this is not technically hard, just a pain in the ass hahaha... Follow up PR?

@CapCap CapCap changed the title [WIP] Transaction Filtering [WIP] More Advanced Transaction Filtering May 31, 2024
@CapCap CapCap force-pushed the transaction_filters branch from 282205f to 1492324 Compare May 31, 2024 18:20
@CapCap CapCap changed the base branch from main to transaction_filter_simple May 31, 2024 18:20
@CapCap CapCap force-pushed the transaction_filters branch from 1492324 to 1891c31 Compare May 31, 2024 18:27
@CapCap CapCap force-pushed the transaction_filters branch from e3ede30 to a74334b Compare June 11, 2024 01:40
@CapCap CapCap marked this pull request as ready for review June 11, 2024 18:21
@CapCap CapCap merged commit 796d8d8 into transaction_filter_simple Jun 11, 2024
4 checks passed
@CapCap CapCap deleted the transaction_filters branch June 11, 2024 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants