Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use benchmark.js #830

Merged
merged 4 commits into from
Aug 19, 2020
Merged

Use benchmark.js #830

merged 4 commits into from
Aug 19, 2020

Conversation

ryanio
Copy link
Contributor

@ryanio ryanio commented Aug 10, 2020

This PR targets merging into #794 to:

  • Use vm.runBlock() instead of looping over txs with vm.runTx()
  • Use benchmark.js to profile runs
  • Use github-action-benchmark to store and visualize data in GH Pages.
  • Reduce fixture file from 50 to 15 blocks (80mb -> 30mb)
    • ci set for 10 blocks i think is sufficient for now

Sample output:

Total number of blocks in data set: 15
Number of blocks to sample: 10
Block 9422905 x 5,458 ops/sec ±3.03% (83 runs sampled)
Block 9422906 x 5,603 ops/sec ±3.14% (90 runs sampled)
Block 9422907 x 5,149 ops/sec ±5.38% (81 runs sampled)
Block 9422908 x 5,526 ops/sec ±0.92% (90 runs sampled)
Block 9422909 x 5,394 ops/sec ±0.72% (87 runs sampled)
Block 9422910 x 4,619 ops/sec ±6.22% (81 runs sampled)
Block 9422911 x 4,944 ops/sec ±2.30% (84 runs sampled)
Block 9422912 x 4,713 ops/sec ±1.66% (83 runs sampled)
Block 9422913 x 4,763 ops/sec ±1.28% (86 runs sampled)
Block 9422914 x 4,536 ops/sec ±1.93% (82 runs sampled)
Block 9422915 x 4,346 ops/sec ±1.98% (79 runs sampled)

Pages link: http://ethereumjs.github.io/ethereumjs-vm/dev/bench/

@ryanio ryanio marked this pull request as draft August 10, 2020 19:17
@codecov
Copy link

codecov bot commented Aug 10, 2020

Codecov Report

Merging #830 into vm/benchmark will decrease coverage by 0.19%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff                @@
##           vm/benchmark     #830      +/-   ##
================================================
- Coverage         84.42%   84.23%   -0.20%     
================================================
  Files                16       17       +1     
  Lines              1239     1224      -15     
  Branches            247      246       -1     
================================================
- Hits               1046     1031      -15     
  Misses              125      125              
  Partials             68       68              
Flag Coverage Δ
#account 92.85% <ø> (ø)
#block 79.97% <ø> (-0.19%) ⬇️
#blockchain 84.50% <ø> (-0.21%) ⬇️
#common 93.60% <ø> (-0.39%) ⬇️
#ethash 85.40% <ø> (ø)
#tx 94.16% <ø> (+0.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
packages/blockchain/src/cache.ts 66.66% <0.00%> (-14.59%) ⬇️
packages/common/src/index.ts 93.52% <0.00%> (-0.40%) ⬇️
packages/tx/src/index.ts 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3942a09...cc7b107. Read the comment docs.

@ryanio ryanio force-pushed the vm/benchmark-js branch 3 times, most recently from 233e851 to b402204 Compare August 10, 2020 20:34
@holgerd77
Copy link
Member

Super cool, the tool suite you figured out with this benchmark action and benchmark.js looks really nice, seems to fit exactly our needs. 😄

@ryanio ryanio marked this pull request as ready for review August 19, 2020 17:09
@ryanio
Copy link
Contributor Author

ryanio commented Aug 19, 2020

This is now ready for review to merge into #794 and then would like to merge into master.

Copy link
Member

@holgerd77 holgerd77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants