Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More reliable VM performance tracking #1204

Closed
holgerd77 opened this issue Apr 13, 2021 · 2 comments
Closed

More reliable VM performance tracking #1204

holgerd77 opened this issue Apr 13, 2021 · 2 comments

Comments

@holgerd77
Copy link
Member

We have the necessary building blocks together for some time now to measure VM performance (derivations) with the introduction of benchmarking in the VM which came into production with #830. The initial measurement of total block runs is definitely interesting but results of such a complex operation turned out to not be reliable enough for detecting performance derivations on CI.

With the additional code separation and benchmarking improvements from #853 we should now (hopefully) be able to create a more deterministic benchmark suite for CI which e.g. might run several repeated atomic operations (benchmarking the StateManager, stack, memory,...), as some first idea (what plays well here still needs to be tested).

This should be at least started on (better to have e.g. just the Stack benchmarking active than nothing) and reintegrated into CI so that a situation of an undetected performance derivation as having happened with the introduction of debug (see #1203) can hopefully be prevented (realistically: for many/the most cases rather than for all cases) in the future.

@holgerd77
Copy link
Member Author

Related: #897 ("Possible way to stabilize CI's benchmark measurements") by @cgewecke

@holgerd77
Copy link
Member Author

We are very much aware of the topic, also have done various integrations, e.g. the EVM profiler integration, will close.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant