Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[subsystem-bench] approval-voting bench: network increased #5463

Closed
AndreiEres opened this issue Aug 26, 2024 · 1 comment · Fixed by #5504
Closed

[subsystem-bench] approval-voting bench: network increased #5463

AndreiEres opened this issue Aug 26, 2024 · 1 comment · Fixed by #5504
Assignees
@alexggh
Labels
T8-polkadot This PR/Issue is related to/affects the Polkadot network. T12-benchmarks This PR/Issue is related to benchmarking and weights.

Comments

@AndreiEres
Copy link
Contributor

AndreiEres commented Aug 26, 2024 •
edited
Loading

image

https://paritytech.github.io/polkadot-sdk/bench/approval-voting-regression-bench/

@alexggh Could you look if it works as expected. If it's ok, we can just adjust our baseline to the new value, otherwise it fails every run in CI.
@AndreiEres AndreiEres added T8-polkadot This PR/Issue is related to/affects the Polkadot network. T12-benchmarks This PR/Issue is related to benchmarking and weights. labels Aug 26, 2024
@alexggh alexggh self-assigned this Aug 27, 2024
@alexggh
Copy link
Contributor

alexggh commented Aug 27, 2024

Could you look if it works as expected. If it's ok, we can just adjust our baseline to the new value, otherwise it fails every run in CI.

Yes, it works as expected if you looks the uber small spike appeared here: #4772 and here #5042, where we basically modified the benchmarks to send a few more messages.

I'll update the benchmark.

@alexggh alexggh mentioned this issue Aug 27, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Aug 28, 2024
@alexggh
Update approval-voting-regression-bench (#5504)
f0fd083 
The accepted divergence rate of 1/1000 is excessive and leads to false
positives especially after
#4772 and
#5042, so let's increase
it to 1/100 since we do have some randomness in the system and there is
no point in being that strict.

Fixes: #5463

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexggh alexggh

Labels
T8-polkadot This PR/Issue is related to/affects the Polkadot network. T12-benchmarks This PR/Issue is related to benchmarking and weights.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update approval-voting-regression-bench paritytech/polkadot-sdk
2 participants
@AndreiEres @alexggh