feat(tools/benchmark): introduce benchmark module (backport #22778) #22851
Conversation
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
(cherry picked from commit 14c841c) # Conflicts: # .github/dependabot.yml # go.work.example # simapp/app.go # simapp/go.mod # simapp/simd/cmd/testnet.go # simapp/v2/go.mod # tests/go.mod
|
Cherry-pick of 14c841c has failed: To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally |
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
github-actions
bot
added
Type: Build
Type: CI
C:server/v2
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| cmd.Printf( | ||
| "success_tx=%d err_tx=%d seq=%d rate=%.2f/s overall=%.2f/s\n", | ||
| successCount, errCount, accSeq, | ||
| float64(successCount-last)/time.Since(since).Seconds(), |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
| "success_tx=%d err_tx=%d seq=%d rate=%.2f/s overall=%.2f/s\n", | ||
| successCount, errCount, accSeq, | ||
| float64(successCount-last)/time.Since(since).Seconds(), | ||
| float64(successCount)/time.Since(begin).Seconds(), |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
| // The operation is one of insert, update, get, or delete. | ||
| // The tx client calls this function to deterministically generate the next operation. | ||
| func (g *Generator) Next() (uint64, *benchmark.Op, error) { | ||
| if g.InsertWeight+g.UpdateWeight+g.GetWeight+g.DeleteWeight != 1 { |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
| // The operation is one of insert, update, get, or delete. | ||
| // The tx client calls this function to deterministically generate the next operation. | ||
| func (g *Generator) Next() (uint64, *benchmark.Op, error) { | ||
| if g.InsertWeight+g.UpdateWeight+g.GetWeight+g.DeleteWeight != 1 { |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
| // The operation is one of insert, update, get, or delete. | ||
| // The tx client calls this function to deterministically generate the next operation. | ||
| func (g *Generator) Next() (uint64, *benchmark.Op, error) { | ||
| if g.InsertWeight+g.UpdateWeight+g.GetWeight+g.DeleteWeight != 1 { |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
| op.Seed = key.Seed() | ||
| op.KeyLength = key.Length() | ||
| op.ValueLength = g.getLength(g.ValueMean, g.ValueStdDev) | ||
| case x < g.InsertWeight+g.UpdateWeight+g.GetWeight: |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
| op.Seed = key.Seed() | ||
| op.KeyLength = key.Length() | ||
| op.ValueLength = g.getLength(g.ValueMean, g.ValueStdDev) | ||
| case x < g.InsertWeight+g.UpdateWeight+g.GetWeight: |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
|
|
||
| // NormUint64 returns a random uint64 with a normal distribution. | ||
| func (g *Generator) NormUint64(mean, stdDev uint64) uint64 { | ||
| return uint64(g.rand.NormFloat64()*float64(stdDev) + float64(mean)) |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
|
|
||
| // NormUint64 returns a random uint64 with a normal distribution. | ||
| func (g *Generator) NormUint64(mean, stdDev uint64) uint64 { | ||
| return uint64(g.rand.NormFloat64()*float64(stdDev) + float64(mean)) |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note test
| package module | ||
|
|
||
| import ( | ||
| "unsafe" |
Check notice
Code scanning / CodeQL
Sensitive package import Note test
Description
Introduces a benchmark module which I wrote to test store performance. It can be used to holistically test the end to end performance of a node. Given an initial configuration tools/benchmark provides:
InitGenesisdistributed acrossnstorekeys, e.g. 20M keys across 5 store keysClient invocation looks like:
On exit it dumps the generator state so that running again should still be in sync. It assumes that any transaction accepted by the network was processed, which may not be the case, so miss rate will probably increase over time. This isn't really a problem for tests.
Obviously this module is built to DOS a node by testing the upper bounds of chain performance; I had massively increase gas limits in tests, so it shouldn't be included by default. It's enabled in simapp and simapp/v2 in this PR though.
closes #750
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
included the correct type prefix in the PR title, you can find examples of the prefixes below:
confirmed
!in the type prefix if API or client breaking changetargeted the correct branch (see PR Targeting)
provided a link to the relevant issue or specification
reviewed "Files changed" and left comments if necessary
included the necessary unit and integration tests
added a changelog entry to
CHANGELOG.mdupdated the relevant documentation or specification, including comments for documenting Go code
confirmed all CI checks have passed
Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.
I have...
Summary by CodeRabbit
Summary by CodeRabbit
Release Notes
New Features
Improvements
Documentation
Tests
Chores
This is an automatic backport of pull request #22778 done by [Mergify](https://mergify.com).