You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have the necessary building blocks together for some time now to measure VM performance (derivations) with the introduction of benchmarking in the VM which came into production with #830. The initial measurement of total block runs is definitely interesting but results of such a complex operation turned out to not be reliable enough for detecting performance derivations on CI.
With the additional code separation and benchmarking improvements from #853 we should now (hopefully) be able to create a more deterministic benchmark suite for CI which e.g. might run several repeated atomic operations (benchmarking the StateManager, stack, memory,...), as some first idea (what plays well here still needs to be tested).
This should be at least started on (better to have e.g. just the Stack benchmarking active than nothing) and reintegrated into CI so that a situation of an undetected performance derivation as having happened with the introduction of debug (see #1203) can hopefully be prevented (realistically: for many/the most cases rather than for all cases) in the future.
The text was updated successfully, but these errors were encountered:
We have the necessary building blocks together for some time now to measure VM performance (derivations) with the introduction of benchmarking in the VM which came into production with #830. The initial measurement of total block runs is definitely interesting but results of such a complex operation turned out to not be reliable enough for detecting performance derivations on CI.
With the additional code separation and benchmarking improvements from #853 we should now (hopefully) be able to create a more deterministic benchmark suite for CI which e.g. might run several repeated atomic operations (benchmarking the StateManager, stack, memory,...), as some first idea (what plays well here still needs to be tested).
This should be at least started on (better to have e.g. just the Stack benchmarking active than nothing) and reintegrated into CI so that a situation of an undetected performance derivation as having happened with the introduction of
debug
(see #1203) can hopefully be prevented (realistically: for many/the most cases rather than for all cases) in the future.The text was updated successfully, but these errors were encountered: