cmd/utils, eth/ethconfig: unindex txs older than ~1 year #22293
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
holiman
approved these changes
Feb 10, 2021
holiman
reviewed
Feb 10, 2021
cmd/utils/flags.go
Outdated
| Name: "txlookuplimit", | ||
| Usage: "Number of recent blocks to maintain transactions index by-hash for (default = index all blocks)", | ||
| Value: 0, | ||
| Usage: "Number of recent blocks to maintain transactions index by-hash for (default = index about one year)", |
There was a problem hiding this comment.
Does 0 mean infinity ? Might need a clarification in the Usage
There was a problem hiding this comment.
Yeah, there's no way to say infinity in Go, so 0 is kind of "disable"
There was a problem hiding this comment.
Right, but 0, for a uint-flag, might to the user mean "Don't index anything". And if the user sets 0 with that notion, it's bad UX
There was a problem hiding this comment.
Alternatives:
- keep it as a
intflag, and make-1mean "everything" - Add to teh Usage string:
(default = index about one year, 0 = index everything)
There was a problem hiding this comment.
Went with the 2nd suggestion
karalabe
force-pushed
the
txunindex-1year
branch
from
7a0fd9f to
409b16e
Compare
9 tasks
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://twitter.com/peter_szilagyi/status/1358767100995784704 .
This PR changes the default number of indexed transaction from infinite to the most recent 1 year (assuming block times of around 13-14s).
The reasoning behind this change is that currently on mainnet there are over 1 billion transactions. Just tracking the
hash->blocknummapping weighs 33GB on the database (requires SSD speeds due to the hash keying) and represents 3/5th of the database entry count, which makes trie operations and compactions exceedingly expensive.Without any form of pruning, the transaction index dataset grows indefinitely. We must bite the bullet and have normal full nodes retain meaningful history, but not indefinite.
Operators still wishing to index all transactions since genesis can run geth with
--txlookuplimit=0. The index can be reconstructed even if it was pruned, so no data is lost. The index is not needed for network health, it's just a local acceleration structure.-