-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: parse chain-id from big genesis file could be slow #18204
Merged
Merged
Changes from 8 commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
192261b
Problem: parse chain-id from big genesis file could be slow
yihuang 7c59903
Update CHANGELOG.md
yihuang ee9b00a
fix unit test
yihuang 567ee2d
patch the library to support early abort
yihuang 55be3ec
add benchmarking
yihuang 9c220f5
mention encoding/json/v2
yihuang c74ba3b
fix error handling
yihuang 7680cdd
fix shadow var
yihuang 1035e32
use simpler solution with different trade off
yihuang 4677221
Merge branch 'main' into parse-chain-id
yihuang be3a3f7
cleanup
yihuang 7cd0787
skip value more efficiently
yihuang c84599c
Merge branch 'main' into parse-chain-id
yihuang df3e2a3
trim space
yihuang 3d7ef1a
trim space
yihuang b09130e
review suggestions
yihuang d52f3b6
assert error messages
yihuang 1ad9a8f
Update types/chain_id.go
yihuang 63b604c
use cronos mainnet genesis for benchmark
yihuang a4cc48a
fix conflicts
yihuang 7a58af1
Merge branch 'main' into parse-chain-id
yihuang 5147d1c
fix cyclic import
yihuang 5d99b23
Merge branch 'main' into parse-chain-id
yihuang ba767d9
fix lint
yihuang 2d3ba07
move directory
yihuang 310b580
Merge branch 'main' into parse-chain-id
yihuang 5145bac
Update x/genutil/types/chain_id.go
yihuang 1bb078f
Merge branch 'main' into parse-chain-id
julienrbrt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we maybe move this to genutil? Imho this makes more sense to have it there as its genesis related than in types. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
package types | ||
|
||
import ( | ||
"errors" | ||
"io" | ||
|
||
"github.com/bcicen/jstream" | ||
yihuang marked this conversation as resolved.
Show resolved
Hide resolved
|
||
) | ||
|
||
const ChainIDFieldName = "chain_id" | ||
|
||
// ParseChainIDFromGenesis parses the chain-id from the genesis file using constant memory. | ||
// | ||
// TODO consider [encoding/json/v2](https://github.com/golang/go/discussions/63397) when it's ready. | ||
func ParseChainIDFromGenesis(reader io.Reader) (string, error) { | ||
decoder := jstream.NewDecoder(reader, 1).EmitKV() | ||
var ( | ||
chain_id string | ||
chain_id_ok bool | ||
yihuang marked this conversation as resolved.
Show resolved
Hide resolved
|
||
) | ||
err := decoder.Decode(func(mv *jstream.MetaValue) bool { | ||
if kv, ok := mv.Value.(jstream.KV); ok { | ||
if kv.Key == ChainIDFieldName { | ||
chain_id, chain_id_ok = kv.Value.(string) | ||
return false | ||
} | ||
} | ||
return true | ||
}) | ||
if len(chain_id) > 0 { | ||
return chain_id, nil | ||
} | ||
if !chain_id_ok { | ||
return "", errors.New("chain-id is not a string") | ||
} | ||
if err == nil { | ||
return "", errors.New("chain-id not found in genesis file") | ||
} | ||
return "", err | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Check failure
Code scanning / gosec
Potential file inclusion via variable