diff --git a/CHANGELOG.md b/CHANGELOG.md index 626151225c..deefb9d12a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## v1.1.11 + +UPGRADE +* [\#927](https://github.com/bnb-chain/bsc/pull/927) add readme for validators about how to enter/exit maintenance +* [\#942](https://github.com/bnb-chain/bsc/pull/942) update the blockNumber of Euler Fork upgrade on BSC Mainnet + + ## v1.1.10 FEATURE diff --git a/README.md b/README.md index c8833941ad..32a0951e08 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ -## Binance Smart Chain +## BNB Smart Chain -The goal of Binance Smart Chain is to bring programmability and interoperability to Binance Chain. In order to embrace the existing popular community and advanced technology, it will bring huge benefits by staying compatible with all the existing smart contracts on Ethereum and Ethereum tooling. And to achieve that, the easiest solution is to develop based on go-ethereum fork, as we respect the great work of Ethereum very much. +The goal of BNB Smart Chain is to bring programmability and interoperability to Binance Chain. In order to embrace the existing popular community and advanced technology, it will bring huge benefits by staying compatible with all the existing smart contracts on Ethereum and Ethereum tooling. And to achieve that, the easiest solution is to develop based on go-ethereum fork, as we respect the great work of Ethereum very much. -Binance Smart Chain starts its development based on go-ethereum fork. So you may see many toolings, binaries and also docs are based on Ethereum ones, such as the name “geth”. +BNB Smart Chain starts its development based on go-ethereum fork. So you may see many toolings, binaries and also docs are based on Ethereum ones, such as the name “geth”. [![API Reference]( https://camo.githubusercontent.com/915b7be44ada53c290eb157634330494ebe3e30a/68747470733a2f2f676f646f632e6f72672f6769746875622e636f6d2f676f6c616e672f6764646f3f7374617475732e737667 @@ -18,7 +18,7 @@ Cross-chain transfer and other communication are possible due to native support - **Interoperable**: Comes with efficient native dual chain communication; Optimized for scaling high-performance dApps that require fast and smooth user experience. - **Distributed with on-chain governance**: Proof of Staked Authority brings in decentralization and community participants. As the native token, BNB will serve as both the gas of smart contract execution and tokens for staking. -More details in [White Paper](http://binance.org/en#smartChain). +More details in [White Paper](https://www.bnbchain.org/en#smartChain). ## Key features @@ -36,7 +36,7 @@ To combine DPoS and PoA for consensus, Binance Smart Chain implement a novel con 2. Validators take turns to produce blocks in a PoA manner, similar to Ethereum's Clique consensus engine. 3. Validator set are elected in and out based on a staking based governance on Binance Chain. 4. The validator set change is relayed via a cross-chain communication mechanism. -5. Parlia consensus engine will interact with a set of [system contracts](https://github.com/binance-chain/docs-site/blob/add-bsc/docs/smart-chain/guides/concepts/system-contract.md) to achieve liveness slash, revenue distributing and validator set renewing func. +5. Parlia consensus engine will interact with a set of [system contracts](https://docs.bnbchain.org/docs/learn/system-contract) to achieve liveness slash, revenue distributing and validator set renewing func. ### Light Client of Binance Chain @@ -44,8 +44,8 @@ To combine DPoS and PoA for consensus, Binance Smart Chain implement a novel con To achieve the cross-chain communication from Binance Chain to Binance Smart Chain, need introduce a on-chain light client verification algorithm. It contains two parts: -1. [Stateless Precompiled contracts](https://github.com/binance-chain/bsc/blob/master/core/vm/contracts_lightclient.go) to do tendermint header verification and Merkle Proof verification. -2. [Stateful solidity contracts](https://github.com/binance-chain/bsc-genesis-contract/blob/master/contracts/TendermintLightClient.sol) to store validator set and trusted appHash. +1. [Stateless Precompiled contracts](https://github.com/bnb-chain/bsc/blob/master/core/vm/contracts_lightclient.go) to do tendermint header verification and Merkle Proof verification. +2. [Stateful solidity contracts](https://github.com/bnb-chain/bsc-genesis-contract/blob/master/contracts/TendermintLightClient.sol) to store validator set and trusted appHash. ## Native Token @@ -124,14 +124,16 @@ This command will: Steps: -1. Download the binary, config and genesis files from [release](https://github.com/bnb-chain/bsc/releases/tag/v1.1.10), or compile the binary by `make geth`. +1. Download the binary, config and genesis files from [release](https://github.com/bnb-chain/bsc/releases/tag/v1.1.11), or compile the binary by `make geth`. 2. Init genesis state: `./geth --datadir node init genesis.json`. 3. Start your fullnode: `./geth --config ./config.toml --datadir ./node`. 4. Or start a validator node: `./geth --config ./config.toml --datadir ./node -unlock ${validatorAddr} --mine --allow-insecure-unlock`. The ${validatorAddr} is the wallet account address of your running validator node. *Note: The default p2p port is 30311 and the RPC port is 8575 which is different from Ethereum.* -More details about [running a node](https://docs.binance.org/smart-chain/developer/fullnode.html) and [becoming a validator](https://docs.binance.org/smart-chain/validator/candidate.html). +More details about [running a node](https://docs.bnbchain.org/docs/validator/fullnode) and [becoming a validator](https://docs.bnbchain.org/docs/validator/testnet/) + + *Note: Although there are some internal protective measures to prevent transactions from crossing over between the main network and test network, you should make sure to always @@ -230,4 +232,4 @@ also included in our repository in the `COPYING.LESSER` file. The bsc binaries (i.e. all code inside of the `cmd` directory) is licensed under the [GNU General Public License v3.0](https://www.gnu.org/licenses/gpl-3.0.en.html), also -included in our repository in the `COPYING` file. \ No newline at end of file +included in our repository in the `COPYING` file. diff --git a/cmd/geth/main.go b/cmd/geth/main.go index e6a59a52fe..9ed921c24f 100644 --- a/cmd/geth/main.go +++ b/cmd/geth/main.go @@ -71,6 +71,8 @@ var ( utils.NoUSBFlag, utils.DirectBroadcastFlag, utils.DisableSnapProtocolFlag, + utils.DisableDiffProtocolFlag, + utils.EnableTrustProtocolFlag, utils.DiffSyncFlag, utils.PipeCommitFlag, utils.RangeLimitFlag, @@ -99,6 +101,7 @@ var ( utils.TxPoolLifetimeFlag, utils.TxPoolReannounceTimeFlag, utils.SyncModeFlag, + utils.TriesVerifyModeFlag, utils.ExitWhenSyncedFlag, utils.GCModeFlag, utils.SnapshotFlag, @@ -126,6 +129,7 @@ var ( utils.CachePreimagesFlag, utils.PersistDiffFlag, utils.DiffBlockFlag, + utils.PruneAncientDataFlag, utils.ListenPortFlag, utils.MaxPeersFlag, utils.MaxPendingPeersFlag, diff --git a/cmd/geth/pruneblock_test.go b/cmd/geth/pruneblock_test.go index 7d78f444fa..78b1d343b5 100644 --- a/cmd/geth/pruneblock_test.go +++ b/cmd/geth/pruneblock_test.go @@ -93,7 +93,7 @@ func testOfflineBlockPruneWithAmountReserved(t *testing.T, amountReserved uint64 t.Fatalf("Failed to back up block: %v", err) } - dbBack, err := rawdb.NewLevelDBDatabaseWithFreezer(chaindbPath, 0, 0, newAncientPath, "", false, true, false) + dbBack, err := rawdb.NewLevelDBDatabaseWithFreezer(chaindbPath, 0, 0, newAncientPath, "", false, true, false, false) if err != nil { t.Fatalf("failed to create database with ancient backend") } @@ -139,7 +139,7 @@ func testOfflineBlockPruneWithAmountReserved(t *testing.T, amountReserved uint64 func BlockchainCreator(t *testing.T, chaindbPath, AncientPath string, blockRemain uint64) (ethdb.Database, []*types.Block, []*types.Block, []types.Receipts, []*big.Int, uint64, *core.BlockChain) { //create a database with ancient freezer - db, err := rawdb.NewLevelDBDatabaseWithFreezer(chaindbPath, 0, 0, AncientPath, "", false, false, false) + db, err := rawdb.NewLevelDBDatabaseWithFreezer(chaindbPath, 0, 0, AncientPath, "", false, false, false, false) if err != nil { t.Fatalf("failed to create database with ancient backend") } diff --git a/cmd/geth/snapshot.go b/cmd/geth/snapshot.go index ab68c11b4d..0d1d3977f4 100644 --- a/cmd/geth/snapshot.go +++ b/cmd/geth/snapshot.go @@ -30,14 +30,17 @@ import ( "github.com/ethereum/go-ethereum/cmd/utils" "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core" "github.com/ethereum/go-ethereum/core/rawdb" "github.com/ethereum/go-ethereum/core/state" "github.com/ethereum/go-ethereum/core/state/pruner" "github.com/ethereum/go-ethereum/core/state/snapshot" "github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/crypto" + "github.com/ethereum/go-ethereum/eth/ethconfig" "github.com/ethereum/go-ethereum/ethdb" "github.com/ethereum/go-ethereum/log" + "github.com/ethereum/go-ethereum/metrics" "github.com/ethereum/go-ethereum/node" "github.com/ethereum/go-ethereum/rlp" "github.com/ethereum/go-ethereum/trie" @@ -133,6 +136,32 @@ geth snapshot verify-state will traverse the whole accounts and storages set based on the specified snapshot and recalculate the root hash of state for verification. In other words, this command does the snapshot to trie conversion. +`, + }, + { + Name: "insecure-prune-all", + Usage: "Prune all trie state data except genesis block, it will break storage for fullnode, only suitable for fast node " + + "who do not need trie storage at all", + ArgsUsage: "", + Action: utils.MigrateFlags(pruneAllState), + Category: "MISCELLANEOUS COMMANDS", + Flags: []cli.Flag{ + utils.DataDirFlag, + utils.AncientFlag, + utils.RopstenFlag, + utils.RinkebyFlag, + utils.GoerliFlag, + }, + Description: ` +will prune all historical trie state data except genesis block. +All trie nodes will be deleted from the database. + +It expects the genesis file as argument. + +WARNING: It's necessary to delete the trie clean cache after the pruning. +If you specify another directory for the trie clean cache via "--cache.trie.journal" +during the use of Geth, please also specify it here for correct deletion. Otherwise +the trie clean cache with default directory will be deleted. `, }, { @@ -228,7 +257,7 @@ func accessDb(ctx *cli.Context, stack *node.Node) (ethdb.Database, error) { } headHeader := headBlock.Header() //Make sure the MPT and snapshot matches before pruning, otherwise the node can not start. - snaptree, err := snapshot.New(chaindb, trie.NewDatabase(chaindb), 256, TriesInMemory, headBlock.Root(), false, false, false) + snaptree, err := snapshot.New(chaindb, trie.NewDatabase(chaindb), 256, TriesInMemory, headBlock.Root(), false, false, false, false) if err != nil { log.Error("snaptree error", "err", err) return nil, err // The relevant snapshot(s) might not exist @@ -396,6 +425,48 @@ func pruneState(ctx *cli.Context) error { return nil } +func pruneAllState(ctx *cli.Context) error { + stack, _ := makeConfigNode(ctx) + defer stack.Close() + + genesisPath := ctx.Args().First() + if len(genesisPath) == 0 { + utils.Fatalf("Must supply path to genesis JSON file") + } + file, err := os.Open(genesisPath) + if err != nil { + utils.Fatalf("Failed to read genesis file: %v", err) + } + defer file.Close() + + g := new(core.Genesis) + if err := json.NewDecoder(file).Decode(g); err != nil { + cfg := gethConfig{ + Eth: ethconfig.Defaults, + Node: defaultNodeConfig(), + Metrics: metrics.DefaultConfig, + } + + // Load config file. + if err := loadConfig(genesisPath, &cfg); err != nil { + utils.Fatalf("%v", err) + } + g = cfg.Eth.Genesis + } + + chaindb := utils.MakeChainDatabase(ctx, stack, false, false) + pruner, err := pruner.NewAllPruner(chaindb) + if err != nil { + log.Error("Failed to open snapshot tree", "err", err) + return err + } + if err = pruner.PruneAll(g); err != nil { + log.Error("Failed to prune state", "err", err) + return err + } + return nil +} + func verifyState(ctx *cli.Context) error { stack, _ := makeConfigNode(ctx) defer stack.Close() @@ -406,7 +477,7 @@ func verifyState(ctx *cli.Context) error { log.Error("Failed to load head block") return errors.New("no head block") } - snaptree, err := snapshot.New(chaindb, trie.NewDatabase(chaindb), 256, 128, headBlock.Root(), false, false, false) + snaptree, err := snapshot.New(chaindb, trie.NewDatabase(chaindb), 256, 128, headBlock.Root(), false, false, false, false) if err != nil { log.Error("Failed to open snapshot tree", "err", err) return err @@ -658,7 +729,7 @@ func dumpState(ctx *cli.Context) error { return err } triesInMemory := ctx.GlobalUint64(utils.TriesInMemoryFlag.Name) - snaptree, err := snapshot.New(db, trie.NewDatabase(db), int(triesInMemory), 256, root, false, false, false) + snaptree, err := snapshot.New(db, trie.NewDatabase(db), int(triesInMemory), 256, root, false, false, false, false) if err != nil { return err } diff --git a/cmd/geth/usage.go b/cmd/geth/usage.go index 7a76d1e321..43e27c5558 100644 --- a/cmd/geth/usage.go +++ b/cmd/geth/usage.go @@ -41,6 +41,8 @@ var AppHelpFlagGroups = []flags.FlagGroup{ utils.NoUSBFlag, utils.DirectBroadcastFlag, utils.DisableSnapProtocolFlag, + utils.DisableDiffProtocolFlag, + utils.EnableTrustProtocolFlag, utils.RangeLimitFlag, utils.SmartCardDaemonPathFlag, utils.NetworkIdFlag, @@ -50,6 +52,7 @@ var AppHelpFlagGroups = []flags.FlagGroup{ utils.RopstenFlag, utils.SepoliaFlag, utils.SyncModeFlag, + utils.TriesVerifyModeFlag, utils.ExitWhenSyncedFlag, utils.GCModeFlag, utils.TxLookupLimitFlag, diff --git a/cmd/utils/flags.go b/cmd/utils/flags.go index 5fc271e262..378a06898b 100644 --- a/cmd/utils/flags.go +++ b/cmd/utils/flags.go @@ -124,6 +124,14 @@ var ( Name: "disablesnapprotocol", Usage: "Disable snap protocol", } + DisableDiffProtocolFlag = cli.BoolFlag{ + Name: "disablediffprotocol", + Usage: "Disable diff protocol", + } + EnableTrustProtocolFlag = cli.BoolFlag{ + Name: "enabletrustprotocol", + Usage: "Enable trust protocol", + } DiffSyncFlag = cli.BoolFlag{ Name: "diffsync", Usage: "Enable diffy sync, Please note that enable diffsync will improve the syncing speed, " + @@ -285,6 +293,20 @@ var ( Usage: "The layer of tries trees that keep in memory", Value: 128, } + defaultVerifyMode = ethconfig.Defaults.TriesVerifyMode + TriesVerifyModeFlag = TextMarshalerFlag{ + Name: "tries-verify-mode", + Usage: `tries verify mode: + "local(default): a normal full node with complete state world(both MPT and snapshot), merkle state root will + be verified against the block header.", + "full: a fast node with only snapshot state world. Merkle state root is verified by the trustworthy remote verify node + by comparing the diffhash(an identify of difflayer generated by the block) and state root.", + "insecure: same as full mode, except that it can tolerate without verifying the diffhash when verify node does not have it.", + "none: no merkle state root verification at all, there is no need to setup or connect remote verify node at all, + it is more light comparing to full and insecure mode, but get a very small chance that the state is not consistent + with other peers."`, + Value: &defaultVerifyMode, + } OverrideBerlinFlag = cli.Uint64Flag{ Name: "override.berlin", Usage: "Manually specify Berlin fork-block, overriding the bundled setting", @@ -478,6 +500,10 @@ var ( Usage: "The number of blocks should be persisted in db (default = 86400)", Value: uint64(86400), } + PruneAncientDataFlag = cli.BoolFlag{ + Name: "pruneancient", + Usage: "Prune ancient data, recommends to the user who don't care about the ancient data. Note that once be turned on, the ancient data will not be recovered again", + } // Miner settings MiningEnabledFlag = cli.BoolFlag{ Name: "mine", @@ -1634,6 +1660,13 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) { if ctx.GlobalIsSet(DiffBlockFlag.Name) { cfg.DiffBlock = ctx.GlobalUint64(DiffBlockFlag.Name) } + if ctx.GlobalIsSet(PruneAncientDataFlag.Name) { + if cfg.SyncMode == downloader.FullSync { + cfg.PruneAncientData = ctx.GlobalBool(PruneAncientDataFlag.Name) + } else { + log.Crit("pruneancient parameter didn't take effect for current syncmode") + } + } if gcmode := ctx.GlobalString(GCModeFlag.Name); gcmode != "full" && gcmode != "archive" { Fatalf("--%s must be either 'full' or 'archive'", GCModeFlag.Name) } @@ -1646,6 +1679,12 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) { if ctx.GlobalIsSet(DisableSnapProtocolFlag.Name) { cfg.DisableSnapProtocol = ctx.GlobalBool(DisableSnapProtocolFlag.Name) } + if ctx.GlobalIsSet(DisableDiffProtocolFlag.Name) { + cfg.DisableDiffProtocol = ctx.GlobalIsSet(DisableDiffProtocolFlag.Name) + } + if ctx.GlobalIsSet(EnableTrustProtocolFlag.Name) { + cfg.EnableTrustProtocol = ctx.GlobalIsSet(EnableTrustProtocolFlag.Name) + } if ctx.GlobalIsSet(DiffSyncFlag.Name) { cfg.DiffSync = ctx.GlobalBool(DiffSyncFlag.Name) } @@ -1679,6 +1718,14 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) { if ctx.GlobalIsSet(TriesInMemoryFlag.Name) { cfg.TriesInMemory = ctx.GlobalUint64(TriesInMemoryFlag.Name) } + if ctx.GlobalIsSet(TriesVerifyModeFlag.Name) { + cfg.TriesVerifyMode = *GlobalTextMarshaler(ctx, TriesVerifyModeFlag.Name).(*core.VerifyMode) + // If a node sets verify mode to full or insecure, it's a fast node and need + // to verify blocks from verify nodes, then it should enable trust protocol. + if cfg.TriesVerifyMode.NeedRemoteVerify() { + cfg.EnableTrustProtocol = true + } + } if ctx.GlobalIsSet(CacheFlag.Name) || ctx.GlobalIsSet(CacheSnapshotFlag.Name) { cfg.SnapshotCache = ctx.GlobalInt(CacheFlag.Name) * ctx.GlobalInt(CacheSnapshotFlag.Name) / 100 } @@ -1714,7 +1761,7 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) { cfg.RPCTxFeeCap = ctx.GlobalFloat64(RPCGlobalTxFeeCapFlag.Name) } if ctx.GlobalIsSet(NoDiscoverFlag.Name) { - cfg.EthDiscoveryURLs, cfg.SnapDiscoveryURLs = []string{}, []string{} + cfg.EthDiscoveryURLs, cfg.SnapDiscoveryURLs, cfg.TrustDiscoveryURLs = []string{}, []string{}, []string{} } else if ctx.GlobalIsSet(DNSDiscoveryFlag.Name) { urls := ctx.GlobalString(DNSDiscoveryFlag.Name) if urls == "" { @@ -1828,6 +1875,7 @@ func SetDNSDiscoveryDefaults(cfg *ethconfig.Config, genesis common.Hash) { if url := params.KnownDNSNetwork(genesis, protocol); url != "" { cfg.EthDiscoveryURLs = []string{url} cfg.SnapDiscoveryURLs = cfg.EthDiscoveryURLs + cfg.TrustDiscoveryURLs = cfg.EthDiscoveryURLs } } @@ -1972,7 +2020,7 @@ func MakeChainDatabase(ctx *cli.Context, stack *node.Node, readonly, disableFree chainDb, err = stack.OpenDatabase(name, cache, handles, "", readonly) } else { name := "chaindata" - chainDb, err = stack.OpenDatabaseWithFreezer(name, cache, handles, ctx.GlobalString(AncientFlag.Name), "", readonly, disableFreeze, false) + chainDb, err = stack.OpenDatabaseWithFreezer(name, cache, handles, ctx.GlobalString(AncientFlag.Name), "", readonly, disableFreeze, false, false) } if err != nil { Fatalf("Could not open database: %v", err) diff --git a/core/block_validator.go b/core/block_validator.go index 12bb61d909..e710d7469a 100644 --- a/core/block_validator.go +++ b/core/block_validator.go @@ -30,23 +30,38 @@ import ( const badBlockCacheExpire = 30 * time.Second +type BlockValidatorOption func(*BlockValidator) *BlockValidator + +func EnableRemoteVerifyManager(remoteValidator *remoteVerifyManager) BlockValidatorOption { + return func(bv *BlockValidator) *BlockValidator { + bv.remoteValidator = remoteValidator + return bv + } +} + // BlockValidator is responsible for validating block headers, uncles and // processed state. // // BlockValidator implements Validator. type BlockValidator struct { - config *params.ChainConfig // Chain configuration options - bc *BlockChain // Canonical block chain - engine consensus.Engine // Consensus engine used for validating + config *params.ChainConfig // Chain configuration options + bc *BlockChain // Canonical block chain + engine consensus.Engine // Consensus engine used for validating + remoteValidator *remoteVerifyManager } // NewBlockValidator returns a new block validator which is safe for re-use -func NewBlockValidator(config *params.ChainConfig, blockchain *BlockChain, engine consensus.Engine) *BlockValidator { +func NewBlockValidator(config *params.ChainConfig, blockchain *BlockChain, engine consensus.Engine, opts ...BlockValidatorOption) *BlockValidator { validator := &BlockValidator{ config: config, engine: engine, bc: blockchain, } + + for _, opt := range opts { + validator = opt(validator) + } + return validator } @@ -92,6 +107,12 @@ func (v *BlockValidator) ValidateBody(block *types.Block) error { } return nil }, + func() error { + if v.remoteValidator != nil && !v.remoteValidator.AncestorVerified(block.Header()) { + return fmt.Errorf("%w, number: %s, hash: %s", ErrAncestorHasNotBeenVerified, block.Number(), block.Hash()) + } + return nil + }, } validateRes := make(chan error, len(validateFuns)) for _, f := range validateFuns { @@ -171,6 +192,10 @@ func (v *BlockValidator) ValidateState(block *types.Block, statedb *state.StateD return err } +func (v *BlockValidator) RemoteVerifyManager() *remoteVerifyManager { + return v.remoteValidator +} + // CalcGasLimit computes the gas limit of the next block after parent. It aims // to keep the baseline gas close to the provided target, and increase it towards // the target if the baseline gas is lower. diff --git a/core/blockchain.go b/core/blockchain.go index 811391ee10..89059792f0 100644 --- a/core/blockchain.go +++ b/core/blockchain.go @@ -28,6 +28,7 @@ import ( "time" lru "github.com/hashicorp/golang-lru" + "golang.org/x/crypto/sha3" "github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common/mclock" @@ -143,6 +144,7 @@ type CacheConfig struct { SnapshotLimit int // Memory allowance (MB) to use for caching snapshot entries in memory Preimages bool // Whether to store preimage of trie key to the disk TriesInMemory uint64 // How many tries keeps in memory + NoTries bool // Insecure settings. Do not have any tries in databases if enabled. SnapshotWait bool // Wait for snapshot construction on startup. TODO(karalabe): This is a dirty hack for testing, nuke it } @@ -163,7 +165,7 @@ var defaultCacheConfig = &CacheConfig{ SnapshotWait: true, } -type BlockChainOption func(*BlockChain) *BlockChain +type BlockChainOption func(*BlockChain) (*BlockChain, error) // BlockChain represents the canonical chain given a database with a genesis // block. The Blockchain manages chain imports, reverts, chain reorganisations. @@ -197,15 +199,16 @@ type BlockChain struct { txLookupLimit uint64 triesInMemory uint64 - hc *HeaderChain - rmLogsFeed event.Feed - chainFeed event.Feed - chainSideFeed event.Feed - chainHeadFeed event.Feed - logsFeed event.Feed - blockProcFeed event.Feed - scope event.SubscriptionScope - genesisBlock *types.Block + hc *HeaderChain + rmLogsFeed event.Feed + chainFeed event.Feed + chainSideFeed event.Feed + chainHeadFeed event.Feed + chainBlockFeed event.Feed + logsFeed event.Feed + blockProcFeed event.Feed + scope event.SubscriptionScope + genesisBlock *types.Block // This mutex synchronizes chain write operations. // Readers don't need to take it, they can just read the database. @@ -227,6 +230,7 @@ type BlockChain struct { // trusted diff layers diffLayerCache *lru.Cache // Cache for the diffLayers diffLayerRLPCache *lru.Cache // Cache for the rlp encoded diffLayers + diffLayerChanCache *lru.Cache // Cache for the difflayer channel diffQueue *prque.Prque // A Priority queue to store recent diff layer diffQueueBuffer chan *types.DiffLayer diffLayerFreezerBlockLimit uint64 @@ -279,6 +283,7 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig *par futureBlocks, _ := lru.New(maxFutureBlocks) diffLayerCache, _ := lru.New(diffLayerCacheLimit) diffLayerRLPCache, _ := lru.New(diffLayerRLPCacheLimit) + diffLayerChanCache, _ := lru.New(diffLayerCacheLimit) bc := &BlockChain{ chainConfig: chainConfig, @@ -289,6 +294,7 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig *par Cache: cacheConfig.TrieCleanLimit, Journal: cacheConfig.TrieCleanJournal, Preimages: cacheConfig.Preimages, + NoTries: cacheConfig.NoTries, }), triesInMemory: cacheConfig.TriesInMemory, quit: make(chan struct{}), @@ -300,6 +306,7 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig *par badBlockCache: badBlockCache, diffLayerCache: diffLayerCache, diffLayerRLPCache: diffLayerRLPCache, + diffLayerChanCache: diffLayerChanCache, txLookupCache: txLookupCache, futureBlocks: futureBlocks, engine: engine, @@ -312,6 +319,7 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig *par diffNumToBlockHashes: make(map[uint64]map[common.Hash]struct{}), diffPeersToDiffHashes: make(map[string]map[common.Hash]struct{}), } + bc.prefetcher = NewStatePrefetcher(chainConfig, bc, engine) bc.forker = NewForkChoice(bc, shouldPreserve) bc.validator = NewBlockValidator(chainConfig, bc, engine) @@ -448,11 +456,16 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig *par log.Warn("Enabling snapshot recovery", "chainhead", head.NumberU64(), "diskbase", *layer) recover = true } - bc.snaps, _ = snapshot.New(bc.db, bc.stateCache.TrieDB(), bc.cacheConfig.SnapshotLimit, int(bc.cacheConfig.TriesInMemory), head.Root(), !bc.cacheConfig.SnapshotWait, true, recover) + bc.snaps, _ = snapshot.New(bc.db, bc.stateCache.TrieDB(), bc.cacheConfig.SnapshotLimit, int(bc.cacheConfig.TriesInMemory), head.Root(), !bc.cacheConfig.SnapshotWait, true, recover, bc.stateCache.NoTries()) } + // write safe point block number + rawdb.WriteSafePointBlockNumber(bc.db, bc.CurrentBlock().NumberU64()) // do options before start any routine for _, option := range options { - bc = option(bc) + bc, err = option(bc) + if err != nil { + return nil, err + } } // Start future block processor. bc.wg.Add(1) @@ -513,11 +526,35 @@ func (bc *BlockChain) cacheReceipts(hash common.Hash, receipts types.Receipts) { bc.receiptsCache.Add(hash, receipts) } -func (bc *BlockChain) cacheDiffLayer(diffLayer *types.DiffLayer) { +func (bc *BlockChain) cacheDiffLayer(diffLayer *types.DiffLayer, diffLayerCh chan struct{}) { + // The difflayer in the system is stored by the map structure, + // so it will be out of order. + // It must be sorted first and then cached, + // otherwise the DiffHash calculated by different nodes will be inconsistent + sort.SliceStable(diffLayer.Codes, func(i, j int) bool { + return diffLayer.Codes[i].Hash.Hex() < diffLayer.Codes[j].Hash.Hex() + }) + sort.SliceStable(diffLayer.Destructs, func(i, j int) bool { + return diffLayer.Destructs[i].Hex() < (diffLayer.Destructs[j].Hex()) + }) + sort.SliceStable(diffLayer.Accounts, func(i, j int) bool { + return diffLayer.Accounts[i].Account.Hex() < diffLayer.Accounts[j].Account.Hex() + }) + sort.SliceStable(diffLayer.Storages, func(i, j int) bool { + return diffLayer.Storages[i].Account.Hex() < diffLayer.Storages[j].Account.Hex() + }) + for index := range diffLayer.Storages { + // Sort keys and vals by key. + sort.Sort(&diffLayer.Storages[index]) + } + if bc.diffLayerCache.Len() >= diffLayerCacheLimit { bc.diffLayerCache.RemoveOldest() } + bc.diffLayerCache.Add(diffLayer.BlockHash, diffLayer) + close(diffLayerCh) + if bc.db.DiffStore() != nil { // push to priority queue before persisting bc.diffQueueBuffer <- diffLayer @@ -559,6 +596,7 @@ func (bc *BlockChain) loadLastState() error { log.Warn("Head block missing, resetting chain", "hash", head) return bc.Reset() } + // Everything seems to be fine, set as the head block bc.currentBlock.Store(currentBlock) headBlockGauge.Update(int64(currentBlock.NumberU64())) @@ -1038,6 +1076,8 @@ func (bc *BlockChain) Stop() { log.Info("Writing cached state to disk", "block", recent.Number(), "hash", recent.Hash(), "root", recent.Root()) if err := triedb.Commit(recent.Root(), true, nil); err != nil { log.Error("Failed to commit recent state trie", "err", err) + } else { + rawdb.WriteSafePointBlockNumber(bc.db, recent.NumberU64()) } } } @@ -1045,6 +1085,8 @@ func (bc *BlockChain) Stop() { log.Info("Writing snapshot state to disk", "root", snapBase) if err := triedb.Commit(snapBase, true, nil); err != nil { log.Error("Failed to commit recent state trie", "err", err) + } else { + rawdb.WriteSafePointBlockNumber(bc.db, bc.CurrentBlock().NumberU64()) } } for !bc.triegc.Empty() { @@ -1482,6 +1524,7 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types. } // Flush an entire trie and restart the counters triedb.Commit(header.Root, true, nil) + rawdb.WriteSafePointBlockNumber(bc.db, chosen) lastWrite = chosen bc.gcproc = 0 } @@ -1512,7 +1555,14 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types. diffLayer.Receipts = receipts diffLayer.BlockHash = block.Hash() diffLayer.Number = block.NumberU64() - bc.cacheDiffLayer(diffLayer) + + diffLayerCh := make(chan struct{}) + if bc.diffLayerChanCache.Len() >= diffLayerCacheLimit { + bc.diffLayerChanCache.RemoveOldest() + } + bc.diffLayerChanCache.Add(diffLayer.BlockHash, diffLayerCh) + + go bc.cacheDiffLayer(diffLayer, diffLayerCh) } wg.Wait() return nil @@ -1836,13 +1886,12 @@ func (bc *BlockChain) insertChain(chain types.Blocks, verifySeals, setHead bool) // Enable prefetching to pull in trie node paths while processing transactions statedb.StartPrefetcher("chain") - var followupInterrupt uint32 + interruptCh := make(chan struct{}) // For diff sync, it may fallback to full sync, so we still do prefetch if len(block.Transactions()) >= prefetchTxNumber { throwaway := statedb.Copy() - go func(start time.Time, followup *types.Block, throwaway *state.StateDB, interrupt *uint32) { - bc.prefetcher.Prefetch(followup, throwaway, bc.vmConfig, &followupInterrupt) - }(time.Now(), block, throwaway, &followupInterrupt) + // do Prefetch in a separate goroutine to avoid blocking the critical path + go bc.prefetcher.Prefetch(block, throwaway, &bc.vmConfig, interruptCh) } //Process block using the parent state as reference point substart := time.Now() @@ -1851,7 +1900,7 @@ func (bc *BlockChain) insertChain(chain types.Blocks, verifySeals, setHead bool) } statedb.SetExpectedStateRoot(block.Root()) statedb, receipts, logs, usedGas, err := bc.processor.Process(block, statedb, bc.vmConfig) - atomic.StoreUint32(&followupInterrupt, 1) + close(interruptCh) // state prefetch can be stopped activeState = statedb if err != nil { bc.reportBlock(block, receipts, err) @@ -1941,6 +1990,7 @@ func (bc *BlockChain) insertChain(chain types.Blocks, verifySeals, setHead bool) stats.processed++ stats.usedGas += usedGas + bc.chainBlockFeed.Send(ChainHeadEvent{block}) dirty, _ := bc.stateCache.TrieDB().Size() stats.report(chain, it.index, dirty) } @@ -2097,6 +2147,9 @@ func (bc *BlockChain) insertSideChain(block *types.Block, it *insertIterator) (i for i := len(hashes) - 1; i >= 0; i-- { // Append the next block to our batch block := bc.GetBlock(hashes[i], numbers[i]) + if block == nil { + log.Crit("Importing heavy sidechain block is nil", "hash", hashes[i], "number", numbers[i]) + } blocks = append(blocks, block) memory += block.Size() @@ -2631,9 +2684,14 @@ func (bc *BlockChain) HandleDiffLayer(diffLayer *types.DiffLayer, pid string, fu return nil } + if diffLayer.DiffHash.Load() == nil { + return fmt.Errorf("unexpected difflayer which diffHash is nil from peeer %s", pid) + } + diffHash := diffLayer.DiffHash.Load().(common.Hash) + bc.diffMux.Lock() defer bc.diffMux.Unlock() - if blockHash, exist := bc.diffHashToBlockHash[diffLayer.DiffHash]; exist && blockHash == diffLayer.BlockHash { + if blockHash, exist := bc.diffHashToBlockHash[diffHash]; exist && blockHash == diffLayer.BlockHash { return nil } @@ -2647,28 +2705,28 @@ func (bc *BlockChain) HandleDiffLayer(diffLayer *types.DiffLayer, pid string, fu return nil } if _, exist := bc.diffPeersToDiffHashes[pid]; exist { - if _, alreadyHas := bc.diffPeersToDiffHashes[pid][diffLayer.DiffHash]; alreadyHas { + if _, alreadyHas := bc.diffPeersToDiffHashes[pid][diffHash]; alreadyHas { return nil } } else { bc.diffPeersToDiffHashes[pid] = make(map[common.Hash]struct{}) } - bc.diffPeersToDiffHashes[pid][diffLayer.DiffHash] = struct{}{} + bc.diffPeersToDiffHashes[pid][diffHash] = struct{}{} if _, exist := bc.diffNumToBlockHashes[diffLayer.Number]; !exist { bc.diffNumToBlockHashes[diffLayer.Number] = make(map[common.Hash]struct{}) } bc.diffNumToBlockHashes[diffLayer.Number][diffLayer.BlockHash] = struct{}{} - if _, exist := bc.diffHashToPeers[diffLayer.DiffHash]; !exist { - bc.diffHashToPeers[diffLayer.DiffHash] = make(map[string]struct{}) + if _, exist := bc.diffHashToPeers[diffHash]; !exist { + bc.diffHashToPeers[diffHash] = make(map[string]struct{}) } - bc.diffHashToPeers[diffLayer.DiffHash][pid] = struct{}{} + bc.diffHashToPeers[diffHash][pid] = struct{}{} if _, exist := bc.blockHashToDiffLayers[diffLayer.BlockHash]; !exist { bc.blockHashToDiffLayers[diffLayer.BlockHash] = make(map[common.Hash]*types.DiffLayer) } - bc.blockHashToDiffLayers[diffLayer.BlockHash][diffLayer.DiffHash] = diffLayer - bc.diffHashToBlockHash[diffLayer.DiffHash] = diffLayer.BlockHash + bc.blockHashToDiffLayers[diffLayer.BlockHash][diffHash] = diffLayer + bc.diffHashToBlockHash[diffHash] = diffLayer.BlockHash return nil } @@ -2869,19 +2927,138 @@ func (bc *BlockChain) InsertHeaderChain(chain []*types.Header, checkFreq int) (i func (bc *BlockChain) TriesInMemory() uint64 { return bc.triesInMemory } // Options -func EnableLightProcessor(bc *BlockChain) *BlockChain { +func EnableLightProcessor(bc *BlockChain) (*BlockChain, error) { bc.processor = NewLightStateProcessor(bc.Config(), bc, bc.engine) - return bc + return bc, nil } -func EnablePipelineCommit(bc *BlockChain) *BlockChain { +func EnablePipelineCommit(bc *BlockChain) (*BlockChain, error) { bc.pipeCommit = true - return bc + return bc, nil } func EnablePersistDiff(limit uint64) BlockChainOption { - return func(chain *BlockChain) *BlockChain { + return func(chain *BlockChain) (*BlockChain, error) { chain.diffLayerFreezerBlockLimit = limit - return chain + return chain, nil } } + +func EnableBlockValidator(chainConfig *params.ChainConfig, engine consensus.Engine, mode VerifyMode, peers verifyPeers) BlockChainOption { + return func(bc *BlockChain) (*BlockChain, error) { + if mode.NeedRemoteVerify() { + vm, err := NewVerifyManager(bc, peers, mode == InsecureVerify) + if err != nil { + return nil, err + } + go vm.mainLoop() + bc.validator = NewBlockValidator(chainConfig, bc, engine, EnableRemoteVerifyManager(vm)) + } + return bc, nil + } +} + +func (bc *BlockChain) GetVerifyResult(blockNumber uint64, blockHash common.Hash, diffHash common.Hash) *VerifyResult { + var res VerifyResult + res.BlockNumber = blockNumber + res.BlockHash = blockHash + + if blockNumber > bc.CurrentHeader().Number.Uint64()+maxDiffForkDist { + res.Status = types.StatusBlockTooNew + return &res + } else if blockNumber > bc.CurrentHeader().Number.Uint64() { + res.Status = types.StatusBlockNewer + return &res + } + + header := bc.GetHeaderByHash(blockHash) + if header == nil { + if blockNumber > bc.CurrentHeader().Number.Uint64()-maxDiffForkDist { + res.Status = types.StatusPossibleFork + return &res + } + + res.Status = types.StatusImpossibleFork + return &res + } + + diff := bc.GetTrustedDiffLayer(blockHash) + if diff != nil { + if diff.DiffHash.Load() == nil { + hash, err := CalculateDiffHash(diff) + if err != nil { + res.Status = types.StatusUnexpectedError + return &res + } + + diff.DiffHash.Store(hash) + } + + if diffHash != diff.DiffHash.Load().(common.Hash) { + res.Status = types.StatusDiffHashMismatch + return &res + } + + res.Status = types.StatusFullVerified + res.Root = header.Root + return &res + } + + res.Status = types.StatusPartiallyVerified + res.Root = header.Root + return &res +} + +func (bc *BlockChain) GetTrustedDiffLayer(blockHash common.Hash) *types.DiffLayer { + var diff *types.DiffLayer + if cached, ok := bc.diffLayerCache.Get(blockHash); ok { + diff = cached.(*types.DiffLayer) + return diff + } + + diffStore := bc.db.DiffStore() + if diffStore != nil { + diff = rawdb.ReadDiffLayer(diffStore, blockHash) + } + return diff +} + +func CalculateDiffHash(d *types.DiffLayer) (common.Hash, error) { + if d == nil { + return common.Hash{}, fmt.Errorf("nil diff layer") + } + + diff := &types.ExtDiffLayer{ + BlockHash: d.BlockHash, + Receipts: make([]*types.ReceiptForStorage, 0), + Number: d.Number, + Codes: d.Codes, + Destructs: d.Destructs, + Accounts: d.Accounts, + Storages: d.Storages, + } + + for index, account := range diff.Accounts { + full, err := snapshot.FullAccount(account.Blob) + if err != nil { + return common.Hash{}, fmt.Errorf("decode full account error: %v", err) + } + // set account root to empty root + diff.Accounts[index].Blob = snapshot.SlimAccountRLP(full.Nonce, full.Balance, common.Hash{}, full.CodeHash) + } + + rawData, err := rlp.EncodeToBytes(diff) + if err != nil { + return common.Hash{}, fmt.Errorf("encode new diff error: %v", err) + } + + hasher := sha3.NewLegacyKeccak256() + _, err = hasher.Write(rawData) + if err != nil { + return common.Hash{}, fmt.Errorf("hasher write error: %v", err) + } + + var hash common.Hash + hasher.Sum(hash[:0]) + return hash, nil +} diff --git a/core/blockchain_diff_test.go b/core/blockchain_diff_test.go index d126ad829e..cae4b917b6 100644 --- a/core/blockchain_diff_test.go +++ b/core/blockchain_diff_test.go @@ -286,7 +286,7 @@ func rawDataToDiffLayer(data rlp.RawValue) (*types.DiffLayer, error) { hasher.Write(data) var diffHash common.Hash hasher.Sum(diffHash[:0]) - diff.DiffHash = diffHash + diff.DiffHash.Store(diffHash) hasher.Reset() return &diff, nil } @@ -484,3 +484,164 @@ func TestGetDiffAccounts(t *testing.T) { } } } + +// newTwoForkedBlockchains returns two blockchains, these two chains are generated by different +// generators, they have some same parent blocks, the number of same blocks are determined by +// testBlocks, once chain1 inserted a non-default block, chain1 and chain2 get forked. +func newTwoForkedBlockchains(len1, len2 int) (chain1 *BlockChain, chain2 *BlockChain) { + signer := types.HomesteadSigner{} + // Create a database pre-initialize with a genesis block + db1 := rawdb.NewMemoryDatabase() + db1.SetDiffStore(memorydb.New()) + (&Genesis{ + Config: params.TestChainConfig, + Alloc: GenesisAlloc{testAddr: {Balance: big.NewInt(100000000000000000)}}, + }).MustCommit(db1) + engine1 := ethash.NewFaker() + chain1, _ = NewBlockChain(db1, nil, params.TestChainConfig, engine1, vm.Config{}, nil, nil, EnablePersistDiff(860000), EnableBlockValidator(params.TestChainConfig, engine1, 0, nil)) + generator1 := func(i int, block *BlockGen) { + // The chain maker doesn't have access to a chain, so the difficulty will be + // lets unset (nil). Set it here to the correct value. + block.SetCoinbase(testAddr) + + for idx, testBlock := range testBlocks { + // Specific block setting, the index in this generator has 1 diff from specified blockNr. + if i+1 == testBlock.blockNr { + for _, testTransaction := range testBlock.txs { + var transaction *types.Transaction + if testTransaction.to == nil { + transaction = types.NewContractCreation(block.TxNonce(testAddr), + testTransaction.value, uint64(commonGas), testTransaction.gasPrice, testTransaction.data) + } else { + transaction = types.NewTransaction(block.TxNonce(testAddr), *testTransaction.to, + testTransaction.value, uint64(commonGas), testTransaction.gasPrice, testTransaction.data) + } + tx, err := types.SignTx(transaction, signer, testKey) + if err != nil { + panic(err) + } + block.AddTxWithChain(chain1, tx) + } + break + } + + // Default block setting. + if idx == len(testBlocks)-1 { + // We want to simulate an empty middle block, having the same state as the + // first one. The last is needs a state change again to force a reorg. + for _, testTransaction := range testBlocks[0].txs { + tx, err := types.SignTx(types.NewTransaction(block.TxNonce(testAddr), *testTransaction.to, + testTransaction.value, uint64(commonGas), testTransaction.gasPrice, testTransaction.data), signer, testKey) + if err != nil { + panic(err) + } + block.AddTxWithChain(chain1, tx) + } + } + } + + } + bs1, _ := GenerateChain(params.TestChainConfig, chain1.Genesis(), ethash.NewFaker(), db1, len1, generator1) + if _, err := chain1.InsertChain(bs1); err != nil { + panic(err) + } + + // Create a database pre-initialize with a genesis block + db2 := rawdb.NewMemoryDatabase() + db2.SetDiffStore(memorydb.New()) + (&Genesis{ + Config: params.TestChainConfig, + Alloc: GenesisAlloc{testAddr: {Balance: big.NewInt(100000000000000000)}}, + }).MustCommit(db2) + engine2 := ethash.NewFaker() + chain2, _ = NewBlockChain(db2, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil, EnablePersistDiff(860000), EnableBlockValidator(params.TestChainConfig, engine2, 0, nil)) + generator2 := func(i int, block *BlockGen) { + // The chain maker doesn't have access to a chain, so the difficulty will be + // lets unset (nil). Set it here to the correct value. + block.SetCoinbase(testAddr) + // We want to simulate an empty middle block, having the same state as the + // first one. The last is needs a state change again to force a reorg. + for _, testTransaction := range testBlocks[0].txs { + tx, err := types.SignTx(types.NewTransaction(block.TxNonce(testAddr), *testTransaction.to, + testTransaction.value, uint64(commonGas), testTransaction.gasPrice, testTransaction.data), signer, testKey) + if err != nil { + panic(err) + } + block.AddTxWithChain(chain1, tx) + } + } + bs2, _ := GenerateChain(params.TestChainConfig, chain2.Genesis(), ethash.NewFaker(), db2, len2, generator2) + if _, err := chain2.InsertChain(bs2); err != nil { + panic(err) + } + + return chain1, chain2 +} + +func testGetRootByDiffHash(t *testing.T, chain1, chain2 *BlockChain, blockNumber uint64, status types.VerifyStatus) { + block2 := chain2.GetBlockByNumber(blockNumber) + if block2 == nil { + t.Fatalf("failed to find block, number: %v", blockNumber) + } + expect := VerifyResult{ + Status: status, + BlockNumber: blockNumber, + BlockHash: block2.Hash(), + } + if status.Code&0xff00 == types.StatusVerified.Code { + expect.Root = block2.Root() + } + + diffLayer2 := chain2.GetTrustedDiffLayer(block2.Hash()) + if diffLayer2 == nil { + t.Fatal("failed to find diff layer") + } + diffHash2 := types.EmptyRootHash + if status != types.StatusDiffHashMismatch { + var err error + diffHash2, err = CalculateDiffHash(diffLayer2) + if err != nil { + t.Fatalf("failed to compute diff hash: %v", err) + } + } + + if status == types.StatusPartiallyVerified { + block1 := chain1.GetBlockByNumber(blockNumber) + if block1 == nil { + t.Fatalf("failed to find block, number: %v", blockNumber) + } + chain1.diffLayerCache.Remove(block1.Hash()) + } + + result := chain1.GetVerifyResult(blockNumber, block2.Hash(), diffHash2) + if result.Status != expect.Status { + t.Fatalf("failed to verify block, number: %v, expect status: %v, real status: %v", blockNumber, expect.Status, result.Status) + } + if result.Root != expect.Root { + t.Fatalf("failed to verify block, number: %v, expect root: %v, real root: %v", blockNumber, expect.Root, result.Root) + } +} + +func TestGetRootByDiffHash(t *testing.T) { + len1 := 23 // length of blockchain1 + len2 := 35 // length of blockchain2 + plen := 11 // length of same parent blocks, which determined by testBlocks. + + chain1, chain2 := newTwoForkedBlockchains(len1, len2) + defer chain1.Stop() + defer chain2.Stop() + + hash1 := chain1.GetBlockByNumber(uint64(plen)).Hash() + hash2 := chain2.GetBlockByNumber(uint64(plen)).Hash() + if hash1 != hash2 { + t.Errorf("chain content mismatch at %d: have hash %v, want hash %v", plen, hash2, hash1) + } + + testGetRootByDiffHash(t, chain1, chain2, 10, types.StatusFullVerified) + testGetRootByDiffHash(t, chain1, chain2, 2, types.StatusPartiallyVerified) + testGetRootByDiffHash(t, chain1, chain2, 10, types.StatusDiffHashMismatch) + testGetRootByDiffHash(t, chain1, chain2, 12, types.StatusImpossibleFork) + testGetRootByDiffHash(t, chain1, chain2, 20, types.StatusPossibleFork) + testGetRootByDiffHash(t, chain1, chain2, 24, types.StatusBlockNewer) + testGetRootByDiffHash(t, chain1, chain2, 35, types.StatusBlockTooNew) +} diff --git a/core/blockchain_notries_test.go b/core/blockchain_notries_test.go new file mode 100644 index 0000000000..7b7a977971 --- /dev/null +++ b/core/blockchain_notries_test.go @@ -0,0 +1,226 @@ +// Copyright 2020 The go-ethereum Authors +// This file is part of the go-ethereum library. +// +// The go-ethereum library is free software: you can redistribute it and/or modify +// it under the terms of the GNU Lesser General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// The go-ethereum library is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Lesser General Public License for more details. +// +// You should have received a copy of the GNU Lesser General Public License +// along with the go-ethereum library. If not, see . + +// Tests that abnormal program termination (i.e.crash) and restart doesn't leave +// the database in some strange state with gaps in the chain, nor with block data +// dangling in the future. + +package core + +import ( + "math/big" + "testing" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/consensus/ethash" + "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/core/vm" + "github.com/ethereum/go-ethereum/ethdb/memorydb" + "github.com/ethereum/go-ethereum/params" +) + +func newMockVerifyPeer() *mockVerifyPeer { + return &mockVerifyPeer{} +} + +type requestRoot struct { + blockNumber uint64 + blockHash common.Hash + diffHash common.Hash +} + +type verifFailedStatus struct { + status types.VerifyStatus + blockNumber uint64 +} + +// mockVerifyPeer is a mocking struct that simulates p2p signals for verification tasks. +type mockVerifyPeer struct { + callback func(*requestRoot) +} + +func (peer *mockVerifyPeer) setCallBack(callback func(*requestRoot)) { + peer.callback = callback +} + +func (peer *mockVerifyPeer) RequestRoot(blockNumber uint64, blockHash common.Hash, diffHash common.Hash) error { + if peer.callback != nil { + peer.callback(&requestRoot{blockNumber, blockHash, diffHash}) + } + return nil +} + +func (peer *mockVerifyPeer) ID() string { + return "mock_peer" +} + +type mockVerifyPeers struct { + peers []VerifyPeer +} + +func (peers *mockVerifyPeers) GetVerifyPeers() []VerifyPeer { + return peers.peers +} + +func newMockRemoteVerifyPeer(peers []VerifyPeer) *mockVerifyPeers { + return &mockVerifyPeers{peers} +} + +func makeTestBackendWithRemoteValidator(blocks int, mode VerifyMode, failed *verifFailedStatus) (*testBackend, *testBackend, []*types.Block, error) { + signer := types.HomesteadSigner{} + + // Create a database pre-initialize with a genesis block + db := rawdb.NewMemoryDatabase() + db.SetDiffStore(memorydb.New()) + (&Genesis{ + Config: params.TestChainConfig, + Alloc: GenesisAlloc{testAddr: {Balance: big.NewInt(100000000000000000)}}, + }).MustCommit(db) + engine := ethash.NewFaker() + + db2 := rawdb.NewMemoryDatabase() + db2.SetDiffStore(memorydb.New()) + (&Genesis{ + Config: params.TestChainConfig, + Alloc: GenesisAlloc{testAddr: {Balance: big.NewInt(100000000000000000)}}, + }).MustCommit(db2) + engine2 := ethash.NewFaker() + + peer := newMockVerifyPeer() + peers := []VerifyPeer{peer} + + verifier, err := NewBlockChain(db, nil, params.TestChainConfig, engine, vm.Config{}, + nil, nil, EnablePersistDiff(100000), EnableBlockValidator(params.TestChainConfig, engine2, LocalVerify, nil)) + if err != nil { + return nil, nil, nil, err + } + + fastnode, err := NewBlockChain(db2, nil, params.TestChainConfig, engine2, vm.Config{}, + nil, nil, EnableBlockValidator(params.TestChainConfig, engine2, mode, newMockRemoteVerifyPeer(peers))) + if err != nil { + return nil, nil, nil, err + } + + generator := func(i int, block *BlockGen) { + // The chain maker doesn't have access to a chain, so the difficulty will be + // lets unset (nil). Set it here to the correct value. + block.SetCoinbase(testAddr) + + for idx, testBlock := range testBlocks { + // Specific block setting, the index in this generator has 1 diff from specified blockNr. + if i+1 == testBlock.blockNr { + for _, testTransaction := range testBlock.txs { + var transaction *types.Transaction + if testTransaction.to == nil { + transaction = types.NewContractCreation(block.TxNonce(testAddr), + testTransaction.value, uint64(commonGas), testTransaction.gasPrice, testTransaction.data) + } else { + transaction = types.NewTransaction(block.TxNonce(testAddr), *testTransaction.to, + testTransaction.value, uint64(commonGas), testTransaction.gasPrice, testTransaction.data) + } + tx, err := types.SignTx(transaction, signer, testKey) + if err != nil { + panic(err) + } + block.AddTxWithChain(verifier, tx) + } + break + } + + // Default block setting. + if idx == len(testBlocks)-1 { + // We want to simulate an empty middle block, having the same state as the + // first one. The last is needs a state change again to force a reorg. + for _, testTransaction := range testBlocks[0].txs { + tx, err := types.SignTx(types.NewTransaction(block.TxNonce(testAddr), *testTransaction.to, + testTransaction.value, uint64(commonGas), testTransaction.gasPrice, testTransaction.data), signer, testKey) + if err != nil { + panic(err) + } + block.AddTxWithChain(verifier, tx) + } + } + } + } + + bs, _ := GenerateChain(params.TestChainConfig, verifier.Genesis(), ethash.NewFaker(), db, blocks, generator) + + peer.setCallBack(func(req *requestRoot) { + if fastnode.validator != nil && fastnode.validator.RemoteVerifyManager() != nil { + resp := verifier.GetVerifyResult(req.blockNumber, req.blockHash, req.diffHash) + if failed != nil && req.blockNumber == failed.blockNumber { + resp.Status = failed.status + } + fastnode.validator.RemoteVerifyManager(). + HandleRootResponse( + resp, peer.ID()) + } + }) + if _, err := verifier.InsertChain(bs); err != nil { + return nil, nil, nil, err + } + + return &testBackend{ + db: db, + chain: verifier, + }, + &testBackend{ + db: db2, + chain: fastnode, + }, bs, nil +} + +func TestFastNode(t *testing.T) { + // test full mode and succeed + _, fastnode, blocks, err := makeTestBackendWithRemoteValidator(10240, FullVerify, nil) + if err != nil { + t.Fatalf(err.Error()) + } + _, err = fastnode.chain.InsertChain(blocks) + if err != nil { + t.Fatalf(err.Error()) + } + // test full mode and failed + failed := &verifFailedStatus{status: types.StatusDiffHashMismatch, blockNumber: 2048} + _, fastnode, blocks, err = makeTestBackendWithRemoteValidator(10240, FullVerify, failed) + if err != nil { + t.Fatalf(err.Error()) + } + _, err = fastnode.chain.InsertChain(blocks) + if err == nil || fastnode.chain.CurrentBlock().NumberU64() != failed.blockNumber+10 { + t.Fatalf("blocks insert should be failed at height %d", failed.blockNumber+11) + } + // test insecure mode and succeed + _, fastnode, blocks, err = makeTestBackendWithRemoteValidator(10240, InsecureVerify, nil) + if err != nil { + t.Fatalf(err.Error()) + } + _, err = fastnode.chain.InsertChain(blocks) + if err != nil { + t.Fatalf(err.Error()) + } + // test insecure mode and failed + failed = &verifFailedStatus{status: types.StatusImpossibleFork, blockNumber: 2048} + _, fastnode, blocks, err = makeTestBackendWithRemoteValidator(10240, FullVerify, failed) + if err != nil { + t.Fatalf(err.Error()) + } + _, err = fastnode.chain.InsertChain(blocks) + if err == nil || fastnode.chain.CurrentBlock().NumberU64() != failed.blockNumber+10 { + t.Fatalf("blocks insert should be failed at height %d", failed.blockNumber+11) + } +} diff --git a/core/blockchain_reader.go b/core/blockchain_reader.go index 5d983d742c..7417ce8b72 100644 --- a/core/blockchain_reader.go +++ b/core/blockchain_reader.go @@ -376,6 +376,11 @@ func (bc *BlockChain) SubscribeChainHeadEvent(ch chan<- ChainHeadEvent) event.Su return bc.scope.Track(bc.chainHeadFeed.Subscribe(ch)) } +// SubscribeChainBlockEvent registers a subscription of ChainBlockEvent. +func (bc *BlockChain) SubscribeChainBlockEvent(ch chan<- ChainHeadEvent) event.Subscription { + return bc.scope.Track(bc.chainBlockFeed.Subscribe(ch)) +} + // SubscribeChainSideEvent registers a subscription of ChainSideEvent. func (bc *BlockChain) SubscribeChainSideEvent(ch chan<- ChainSideEvent) event.Subscription { return bc.scope.Track(bc.chainSideFeed.Subscribe(ch)) diff --git a/core/blockchain_repair_test.go b/core/blockchain_repair_test.go index 3b161c1421..658c9a9d09 100644 --- a/core/blockchain_repair_test.go +++ b/core/blockchain_repair_test.go @@ -1762,7 +1762,7 @@ func testRepair(t *testing.T, tt *rewindTest, snapshots bool) { } os.RemoveAll(datadir) - db, err := rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false) + db, err := rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false, false) if err != nil { t.Fatalf("Failed to create persistent database: %v", err) } @@ -1833,7 +1833,7 @@ func testRepair(t *testing.T, tt *rewindTest, snapshots bool) { db.Close() // Start a new blockchain back up and see where the repait leads us - db, err = rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false) + db, err = rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false, false) if err != nil { t.Fatalf("Failed to reopen persistent database: %v", err) } @@ -1892,7 +1892,7 @@ func TestIssue23496(t *testing.T) { } os.RemoveAll(datadir) - db, err := rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false) + db, err := rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false, false) if err != nil { t.Fatalf("Failed to create persistent database: %v", err) } @@ -1949,7 +1949,7 @@ func TestIssue23496(t *testing.T) { db.Close() // Start a new blockchain back up and see where the repair leads us - db, err = rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false) + db, err = rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false, false) if err != nil { t.Fatalf("Failed to reopen persistent database: %v", err) } diff --git a/core/blockchain_sethead_test.go b/core/blockchain_sethead_test.go index ac89b6f375..8cd87fd6db 100644 --- a/core/blockchain_sethead_test.go +++ b/core/blockchain_sethead_test.go @@ -1961,7 +1961,7 @@ func testSetHead(t *testing.T, tt *rewindTest, snapshots bool) { } os.RemoveAll(datadir) - db, err := rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false) + db, err := rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false, false) if err != nil { t.Fatalf("Failed to create persistent database: %v", err) } diff --git a/core/blockchain_snapshot_test.go b/core/blockchain_snapshot_test.go index 9ccb2c7d43..23f4ff9af6 100644 --- a/core/blockchain_snapshot_test.go +++ b/core/blockchain_snapshot_test.go @@ -65,7 +65,7 @@ func (basic *snapshotTestBasic) prepare(t *testing.T) (*BlockChain, []*types.Blo } os.RemoveAll(datadir) - db, err := rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false) + db, err := rawdb.NewLevelDBDatabaseWithFreezer(datadir, 0, 0, datadir, "", false, false, false, false) if err != nil { t.Fatalf("Failed to create persistent database: %v", err) } @@ -249,7 +249,7 @@ func (snaptest *crashSnapshotTest) test(t *testing.T) { db.Close() // Start a new blockchain back up and see where the repair leads us - newdb, err := rawdb.NewLevelDBDatabaseWithFreezer(snaptest.datadir, 0, 0, snaptest.datadir, "", false, false, false) + newdb, err := rawdb.NewLevelDBDatabaseWithFreezer(snaptest.datadir, 0, 0, snaptest.datadir, "", false, false, false, false) if err != nil { t.Fatalf("Failed to reopen persistent database: %v", err) } diff --git a/core/blockchain_test.go b/core/blockchain_test.go index bc0d57e085..83aae79390 100644 --- a/core/blockchain_test.go +++ b/core/blockchain_test.go @@ -264,7 +264,7 @@ func TestBlockImportVerification(t *testing.T) { } defer processor.Stop() // Start fork from current height - processor = EnablePipelineCommit(processor) + processor, _ = EnablePipelineCommit(processor) testInvalidStateRootBlockImport(t, processor, length, 10, true) } @@ -903,7 +903,7 @@ func TestFastVsFullChains(t *testing.T) { t.Fatalf("failed to create temp freezer dir: %v", err) } defer os.Remove(frdir) - ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false) + ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -999,7 +999,7 @@ func TestLightVsFastVsFullChainHeads(t *testing.T) { t.Fatalf("failed to create temp freezer dir: %v", err) } defer os.Remove(dir) - db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), dir, "", false, false, false) + db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), dir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -1866,7 +1866,7 @@ func TestBlockchainRecovery(t *testing.T) { } defer os.Remove(frdir) - ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false) + ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -1937,7 +1937,7 @@ func TestInsertReceiptChainRollback(t *testing.T) { t.Fatalf("failed to create temp freezer dir: %v", err) } defer os.Remove(frdir) - ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false) + ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -2203,7 +2203,7 @@ func testInsertKnownChainData(t *testing.T, typ string) { t.Fatalf("failed to create temp freezer dir: %v", err) } defer os.Remove(dir) - chaindb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), dir, "", false, false, false) + chaindb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), dir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -2367,7 +2367,7 @@ func testInsertKnownChainDataWithMerging(t *testing.T, typ string, mergeHeight i t.Fatalf("failed to create temp freezer dir: %v", err) } defer os.Remove(dir) - chaindb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), dir, "", false, false, false) + chaindb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), dir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -2677,7 +2677,7 @@ func TestTransactionIndices(t *testing.T) { t.Fatalf("failed to create temp freezer dir: %v", err) } defer os.Remove(frdir) - ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false) + ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -2705,7 +2705,7 @@ func TestTransactionIndices(t *testing.T) { // Init block chain with external ancients, check all needed indices has been indexed. limit := []uint64{0, 32, 64, 128} for _, l := range limit { - ancientDb, err = rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false) + ancientDb, err = rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -2725,7 +2725,7 @@ func TestTransactionIndices(t *testing.T) { } // Reconstruct a block chain which only reserves HEAD-64 tx indices - ancientDb, err = rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false) + ancientDb, err = rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } @@ -2804,7 +2804,7 @@ func TestSkipStaleTxIndicesInSnapSync(t *testing.T) { t.Fatalf("failed to create temp freezer dir: %v", err) } defer os.Remove(frdir) - ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false) + ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create temp freezer db: %v", err) } diff --git a/core/error.go b/core/error.go index 0cc61b7a55..c4ba1e9d37 100644 --- a/core/error.go +++ b/core/error.go @@ -37,6 +37,12 @@ var ( // ErrDiffLayerNotFound is returned when diff layer not found. ErrDiffLayerNotFound = errors.New("diff layer not found") + // ErrDiffLayerNotFound is returned when block - 11 has not been verified by the remote verifier. + ErrAncestorHasNotBeenVerified = errors.New("block ancestor has not been verified") + + // ErrCurrentBlockNotFound is returned when current block not found. + ErrCurrentBlockNotFound = errors.New("current block not found") + // ErrKnownBadBlock is return when the block is a known bad block ErrKnownBadBlock = errors.New("already known bad block") ) diff --git a/core/rawdb/accessors_chain_test.go b/core/rawdb/accessors_chain_test.go index cbbd99c9c6..7b6b5dca29 100644 --- a/core/rawdb/accessors_chain_test.go +++ b/core/rawdb/accessors_chain_test.go @@ -441,7 +441,7 @@ func TestAncientStorage(t *testing.T) { } defer os.RemoveAll(frdir) - db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false) + db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create database with ancient backend") } @@ -582,7 +582,7 @@ func BenchmarkWriteAncientBlocks(b *testing.B) { b.Fatalf("failed to create temp freezer dir: %v", err) } defer os.RemoveAll(frdir) - db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false) + db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { b.Fatalf("failed to create database with ancient backend") } @@ -892,7 +892,7 @@ func TestHeadersRLPStorage(t *testing.T) { } defer os.Remove(frdir) - db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false) + db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false) if err != nil { t.Fatalf("failed to create database with ancient backend") } diff --git a/core/rawdb/accessors_metadata.go b/core/rawdb/accessors_metadata.go index 3b0fcf0f2d..6aa4427899 100644 --- a/core/rawdb/accessors_metadata.go +++ b/core/rawdb/accessors_metadata.go @@ -18,6 +18,7 @@ package rawdb import ( "encoding/json" + "math/big" "time" "github.com/ethereum/go-ethereum/common" @@ -27,6 +28,12 @@ import ( "github.com/ethereum/go-ethereum/rlp" ) +// FreezerType enumerator +const ( + EntireFreezerType uint64 = iota // classic ancient type + PruneFreezerType // prune ancient type +) + // ReadDatabaseVersion retrieves the version number of the database. func ReadDatabaseVersion(db ethdb.KeyValueReader) *uint64 { var version uint64 @@ -173,3 +180,83 @@ func WriteTransitionStatus(db ethdb.KeyValueWriter, data []byte) { log.Crit("Failed to store the eth2 transition status", "err", err) } } + +// ReadOffSetOfCurrentAncientFreezer return prune block start +func ReadOffSetOfCurrentAncientFreezer(db ethdb.KeyValueReader) uint64 { + offset, _ := db.Get(offSetOfCurrentAncientFreezer) + if offset == nil { + return 0 + } + return new(big.Int).SetBytes(offset).Uint64() +} + +// WriteOffSetOfCurrentAncientFreezer write prune block start +func WriteOffSetOfCurrentAncientFreezer(db ethdb.KeyValueWriter, offset uint64) { + if err := db.Put(offSetOfCurrentAncientFreezer, new(big.Int).SetUint64(offset).Bytes()); err != nil { + log.Crit("Failed to store offSetOfAncientFreezer", "err", err) + } +} + +// ReadOffSetOfLastAncientFreezer return last prune block start +func ReadOffSetOfLastAncientFreezer(db ethdb.KeyValueReader) uint64 { + offset, _ := db.Get(offSetOfLastAncientFreezer) + if offset == nil { + return 0 + } + return new(big.Int).SetBytes(offset).Uint64() +} + +// WriteOffSetOfLastAncientFreezer wirte before prune block start +func WriteOffSetOfLastAncientFreezer(db ethdb.KeyValueWriter, offset uint64) { + if err := db.Put(offSetOfLastAncientFreezer, new(big.Int).SetUint64(offset).Bytes()); err != nil { + log.Crit("Failed to store offSetOfAncientFreezer", "err", err) + } +} + +// ReadFrozenOfAncientFreezer return freezer block number +func ReadFrozenOfAncientFreezer(db ethdb.KeyValueReader) uint64 { + fozen, _ := db.Get(frozenOfAncientDBKey) + if fozen == nil { + return 0 + } + return new(big.Int).SetBytes(fozen).Uint64() +} + +// WriteFrozenOfAncientFreezer write freezer block number +func WriteFrozenOfAncientFreezer(db ethdb.KeyValueWriter, frozen uint64) { + if err := db.Put(frozenOfAncientDBKey, new(big.Int).SetUint64(frozen).Bytes()); err != nil { + log.Crit("Failed to store offSetOfAncientFreezer", "err", err) + } +} + +// ReadSafePointBlockNumber return the number of block that roothash save to disk +func ReadSafePointBlockNumber(db ethdb.KeyValueReader) uint64 { + num, _ := db.Get(LastSafePointBlockKey) + if num == nil { + return 0 + } + return new(big.Int).SetBytes(num).Uint64() +} + +// WriteSafePointBlockNumber write the number of block that roothash save to disk +func WriteSafePointBlockNumber(db ethdb.KeyValueWriter, number uint64) { + if err := db.Put(LastSafePointBlockKey, new(big.Int).SetUint64(number).Bytes()); err != nil { + log.Crit("Failed to store offSetOfAncientFreezer", "err", err) + } +} + +// ReadAncientType return freezer type +func ReadAncientType(db ethdb.KeyValueReader) uint64 { + data, _ := db.Get(pruneAncientKey) + if data == nil { + return EntireFreezerType + } + return new(big.Int).SetBytes(data).Uint64() +} + +// WriteAncientType write freezer type +func WriteAncientType(db ethdb.KeyValueWriter, flag uint64) { + if err := db.Put(pruneAncientKey, new(big.Int).SetUint64(flag).Bytes()); err != nil { + log.Crit("Failed to store offSetOfAncientFreezer", "err", err) + } +} diff --git a/core/rawdb/chain_iterator.go b/core/rawdb/chain_iterator.go index 7c272492e8..13e684a69c 100644 --- a/core/rawdb/chain_iterator.go +++ b/core/rawdb/chain_iterator.go @@ -186,6 +186,9 @@ func iterateTransactions(db ethdb.Database, from uint64, to uint64, reverse bool // signal received. func indexTransactions(db ethdb.Database, from uint64, to uint64, interrupt chan struct{}, hook func(uint64) bool) { // short circuit for invalid range + if offset := db.AncientOffSet(); offset > from { + from = offset + } if from >= to { return } @@ -278,6 +281,9 @@ func indexTransactionsForTesting(db ethdb.Database, from uint64, to uint64, inte // signal received. func unindexTransactions(db ethdb.Database, from uint64, to uint64, interrupt chan struct{}, hook func(uint64) bool) { // short circuit for invalid range + if offset := db.AncientOffSet(); offset > from { + from = offset + } if from >= to { return } diff --git a/core/rawdb/database.go b/core/rawdb/database.go index 4b15ab46fa..7dd5b47ff2 100644 --- a/core/rawdb/database.go +++ b/core/rawdb/database.go @@ -20,7 +20,6 @@ import ( "bytes" "errors" "fmt" - "math/big" "os" "sync/atomic" "time" @@ -178,33 +177,6 @@ func NewDatabase(db ethdb.KeyValueStore) ethdb.Database { return &nofreezedb{KeyValueStore: db} } -func ReadOffSetOfCurrentAncientFreezer(db ethdb.KeyValueReader) uint64 { - offset, _ := db.Get(offSetOfCurrentAncientFreezer) - if offset == nil { - return 0 - } - return new(big.Int).SetBytes(offset).Uint64() -} - -func ReadOffSetOfLastAncientFreezer(db ethdb.KeyValueReader) uint64 { - offset, _ := db.Get(offSetOfLastAncientFreezer) - if offset == nil { - return 0 - } - return new(big.Int).SetBytes(offset).Uint64() -} - -func WriteOffSetOfCurrentAncientFreezer(db ethdb.KeyValueWriter, offset uint64) { - if err := db.Put(offSetOfCurrentAncientFreezer, new(big.Int).SetUint64(offset).Bytes()); err != nil { - log.Crit("Failed to store offSetOfAncientFreezer", "err", err) - } -} -func WriteOffSetOfLastAncientFreezer(db ethdb.KeyValueWriter, offset uint64) { - if err := db.Put(offSetOfLastAncientFreezer, new(big.Int).SetUint64(offset).Bytes()); err != nil { - log.Crit("Failed to store offSetOfAncientFreezer", "err", err) - } -} - // NewFreezerDb only create a freezer without statedb. func NewFreezerDb(db ethdb.KeyValueStore, frz, namespace string, readonly bool, newOffSet uint64) (*freezer, error) { // Create the idle freezer instance, this operation should be atomic to avoid mismatch between offset and acientDB. @@ -220,7 +192,29 @@ func NewFreezerDb(db ethdb.KeyValueStore, frz, namespace string, readonly bool, // NewDatabaseWithFreezer creates a high level database on top of a given key- // value data store with a freezer moving immutable chain segments into cold // storage. -func NewDatabaseWithFreezer(db ethdb.KeyValueStore, freezer string, namespace string, readonly, disableFreeze, isLastOffset bool) (ethdb.Database, error) { +func NewDatabaseWithFreezer(db ethdb.KeyValueStore, freezer string, namespace string, readonly, disableFreeze, isLastOffset, pruneAncientData bool) (ethdb.Database, error) { + if pruneAncientData && !disableFreeze && !readonly { + frdb, err := newPrunedFreezer(freezer, db) + if err != nil { + return nil, err + } + + go frdb.freeze() + WriteAncientType(db, PruneFreezerType) + return &freezerdb{ + KeyValueStore: db, + AncientStore: frdb, + }, nil + } + + if pruneAncientData { + log.Error("pruneancient not take effect, disableFreezer or readonly be set") + } + + if ReadAncientType(db) == PruneFreezerType { + log.Warn("prune ancinet flag is set, may start fail, can add pruneancient parameter resolve") + } + // Create the idle freezer instance frdb, err := newFreezer(freezer, namespace, readonly, freezerTableSize, FreezerNoSnappy) if err != nil { @@ -308,7 +302,8 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, freezer string, namespace st // feezer. } } - + // no prune ancinet start success + WriteAncientType(db, EntireFreezerType) // Freezer is consistent with the key-value database, permit combining the two if !disableFreeze && !frdb.readonly { frdb.wg.Add(1) @@ -348,12 +343,12 @@ func NewLevelDBDatabase(file string, cache int, handles int, namespace string, r // NewLevelDBDatabaseWithFreezer creates a persistent key-value database with a // freezer moving immutable chain segments into cold storage. -func NewLevelDBDatabaseWithFreezer(file string, cache int, handles int, freezer string, namespace string, readonly, disableFreeze, isLastOffset bool) (ethdb.Database, error) { +func NewLevelDBDatabaseWithFreezer(file string, cache int, handles int, freezer string, namespace string, readonly, disableFreeze, isLastOffset, pruneAncientData bool) (ethdb.Database, error) { kvdb, err := leveldb.New(file, cache, handles, namespace, readonly) if err != nil { return nil, err } - frdb, err := NewDatabaseWithFreezer(kvdb, freezer, namespace, readonly, disableFreeze, isLastOffset) + frdb, err := NewDatabaseWithFreezer(kvdb, freezer, namespace, readonly, disableFreeze, isLastOffset, pruneAncientData) if err != nil { kvdb.Close() return nil, err diff --git a/core/rawdb/freezer.go b/core/rawdb/freezer.go index 18e610ffca..0a4c8f9b43 100644 --- a/core/rawdb/freezer.go +++ b/core/rawdb/freezer.go @@ -399,9 +399,7 @@ func (f *freezer) freeze(db ethdb.KeyValueStore) { } select { case <-time.NewTimer(freezerRecheckInterval).C: - backoff = false case triggered = <-f.trigger: - backoff = false case <-f.quit: return } @@ -460,84 +458,9 @@ func (f *freezer) freeze(db ethdb.KeyValueStore) { log.Crit("Failed to flush frozen tables", "err", err) } - // Wipe out all data from the active database - batch := db.NewBatch() - for i := 0; i < len(ancients); i++ { - // Always keep the genesis block in active database - if first+uint64(i) != 0 { - DeleteBlockWithoutNumber(batch, ancients[i], first+uint64(i)) - DeleteCanonicalHash(batch, first+uint64(i)) - } - } - if err := batch.Write(); err != nil { - log.Crit("Failed to delete frozen canonical blocks", "err", err) - } - batch.Reset() - - // Wipe out side chains also and track dangling side chains - var dangling []common.Hash - for number := first; number < f.frozen; number++ { - // Always keep the genesis block in active database - if number != 0 { - dangling = ReadAllHashes(db, number) - for _, hash := range dangling { - log.Trace("Deleting side chain", "number", number, "hash", hash) - DeleteBlock(batch, hash, number) - } - } - } - if err := batch.Write(); err != nil { - log.Crit("Failed to delete frozen side blocks", "err", err) - } - batch.Reset() - - // Step into the future and delete and dangling side chains - if f.frozen > 0 { - tip := f.frozen - for len(dangling) > 0 { - drop := make(map[common.Hash]struct{}) - for _, hash := range dangling { - log.Debug("Dangling parent from freezer", "number", tip-1, "hash", hash) - drop[hash] = struct{}{} - } - children := ReadAllHashes(db, tip) - for i := 0; i < len(children); i++ { - // Dig up the child and ensure it's dangling - child := ReadHeader(nfdb, children[i], tip) - if child == nil { - log.Error("Missing dangling header", "number", tip, "hash", children[i]) - continue - } - if _, ok := drop[child.ParentHash]; !ok { - children = append(children[:i], children[i+1:]...) - i-- - continue - } - // Delete all block data associated with the child - log.Debug("Deleting dangling block", "number", tip, "hash", children[i], "parent", child.ParentHash) - DeleteBlock(batch, children[i], tip) - } - dangling = children - tip++ - } - if err := batch.Write(); err != nil { - log.Crit("Failed to delete dangling side blocks", "err", err) - } - } - - // Log something friendly for the user - context := []interface{}{ - "blocks", f.frozen - first, "elapsed", common.PrettyDuration(time.Since(start)), "number", f.frozen - 1, - } - if n := len(ancients); n > 0 { - context = append(context, []interface{}{"hash", ancients[n-1]}...) - } - log.Info("Deep froze chain segment", context...) - - // Avoid database thrashing with tiny writes - if f.frozen-first < freezerBatchLimit { - backoff = true - } + // Batch of blocks have been frozen, flush them before wiping from leveldb + backoff = f.frozen-first >= freezerBatchLimit + gcKvStore(db, ancients, first, f.frozen, start) } } @@ -592,3 +515,81 @@ func (f *freezer) freezeRange(nfdb *nofreezedb, number, limit uint64) (hashes [] return hashes, err } + +// delete leveldb data that save to ancientdb, split from func freeze +func gcKvStore(db ethdb.KeyValueStore, ancients []common.Hash, first uint64, frozen uint64, start time.Time) { + // Wipe out all data from the active database + batch := db.NewBatch() + for i := 0; i < len(ancients); i++ { + // Always keep the genesis block in active database + if blockNumber := first + uint64(i); blockNumber != 0 { + DeleteBlockWithoutNumber(batch, ancients[i], blockNumber) + DeleteCanonicalHash(batch, blockNumber) + } + } + if err := batch.Write(); err != nil { + log.Crit("Failed to delete frozen canonical blocks", "err", err) + } + batch.Reset() + + // Wipe out side chains also and track dangling side chians + var dangling []common.Hash + for number := first; number < frozen; number++ { + // Always keep the genesis block in active database + if number != 0 { + dangling = ReadAllHashes(db, number) + for _, hash := range dangling { + log.Trace("Deleting side chain", "number", number, "hash", hash) + DeleteBlock(batch, hash, number) + } + } + } + if err := batch.Write(); err != nil { + log.Crit("Failed to delete frozen side blocks", "err", err) + } + batch.Reset() + + // Step into the future and delete and dangling side chains + if frozen > 0 { + tip := frozen + nfdb := &nofreezedb{KeyValueStore: db} + for len(dangling) > 0 { + drop := make(map[common.Hash]struct{}) + for _, hash := range dangling { + log.Debug("Dangling parent from freezer", "number", tip-1, "hash", hash) + drop[hash] = struct{}{} + } + children := ReadAllHashes(db, tip) + for i := 0; i < len(children); i++ { + // Dig up the child and ensure it's dangling + child := ReadHeader(nfdb, children[i], tip) + if child == nil { + log.Error("Missing dangling header", "number", tip, "hash", children[i]) + continue + } + if _, ok := drop[child.ParentHash]; !ok { + children = append(children[:i], children[i+1:]...) + i-- + continue + } + // Delete all block data associated with the child + log.Debug("Deleting dangling block", "number", tip, "hash", children[i], "parent", child.ParentHash) + DeleteBlock(batch, children[i], tip) + } + dangling = children + tip++ + } + if err := batch.Write(); err != nil { + log.Crit("Failed to delete dangling side blocks", "err", err) + } + } + + // Log something friendly for the user + context := []interface{}{ + "blocks", frozen - first, "elapsed", common.PrettyDuration(time.Since(start)), "number", frozen - 1, + } + if n := len(ancients); n > 0 { + context = append(context, []interface{}{"hash", ancients[n-1]}...) + } + log.Info("Deep froze chain segment", context...) +} diff --git a/core/rawdb/prunedfreezer.go b/core/rawdb/prunedfreezer.go new file mode 100644 index 0000000000..453fe7d261 --- /dev/null +++ b/core/rawdb/prunedfreezer.go @@ -0,0 +1,297 @@ +package rawdb + +import ( + "math" + "os" + "path/filepath" + "sync" + "sync/atomic" + "time" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/ethdb" + "github.com/ethereum/go-ethereum/log" + "github.com/ethereum/go-ethereum/params" + "github.com/prometheus/tsdb/fileutil" +) + +// prunedfreezer not contain ancient data, only record 'frozen' , the next recycle block number form kvstore. +type prunedfreezer struct { + db ethdb.KeyValueStore // Meta database + // WARNING: The `frozen` field is accessed atomically. On 32 bit platforms, only + // 64-bit aligned fields can be atomic. The struct is guaranteed to be so aligned, + // so take advantage of that (https://golang.org/pkg/sync/atomic/#pkg-note-BUG). + frozen uint64 // BlockNumber of next frozen block + threshold uint64 // Number of recent blocks not to freeze (params.FullImmutabilityThreshold apart from tests) + + instanceLock fileutil.Releaser // File-system lock to prevent double opens + quit chan struct{} + closeOnce sync.Once +} + +// newNoDataFreezer creates a chain freezer that deletes data enough ‘old’. +func newPrunedFreezer(datadir string, db ethdb.KeyValueStore) (*prunedfreezer, error) { + if info, err := os.Lstat(datadir); !os.IsNotExist(err) { + if info.Mode()&os.ModeSymlink != 0 { + log.Warn("Symbolic link ancient database is not supported", "path", datadir) + return nil, errSymlinkDatadir + } + } + + lock, _, err := fileutil.Flock(filepath.Join(datadir, "../NODATA_ANCIENT_FLOCK")) + if err != nil { + return nil, err + } + + freezer := &prunedfreezer{ + db: db, + threshold: params.FullImmutabilityThreshold, + instanceLock: lock, + quit: make(chan struct{}), + } + + if err := freezer.repair(datadir); err != nil { + return nil, err + } + + log.Info("Opened ancientdb with nodata mode", "database", datadir, "frozen", freezer.frozen) + return freezer, nil +} + +// repair init frozen , compatible disk-ancientdb and pruner-block-tool. +func (f *prunedfreezer) repair(datadir string) error { + // compatible prune-block-tool + offset := ReadOffSetOfCurrentAncientFreezer(f.db) + + // compatible freezer + min := uint64(math.MaxUint64) + for name, disableSnappy := range FreezerNoSnappy { + table, err := NewFreezerTable(datadir, name, disableSnappy, true) + if err != nil { + return err + } + items := atomic.LoadUint64(&table.items) + if min > items { + min = items + } + table.Close() + } + offset += min + + if frozen := ReadFrozenOfAncientFreezer(f.db); frozen > offset { + offset = frozen + } + + atomic.StoreUint64(&f.frozen, offset) + if err := f.Sync(); err != nil { + return nil + } + return nil +} + +// Close terminates the chain prunedfreezer. +func (f *prunedfreezer) Close() error { + var err error + f.closeOnce.Do(func() { + close(f.quit) + f.Sync() + err = f.instanceLock.Release() + }) + return err +} + +// HasAncient returns an indicator whether the specified ancient data exists, return nil. +func (f *prunedfreezer) HasAncient(kind string, number uint64) (bool, error) { + return false, nil +} + +// Ancient retrieves an ancient binary blob from prunedfreezer, return nil. +func (f *prunedfreezer) Ancient(kind string, number uint64) ([]byte, error) { + if _, ok := FreezerNoSnappy[kind]; ok { + if number >= atomic.LoadUint64(&f.frozen) { + return nil, errOutOfBounds + } + return nil, nil + } + return nil, errUnknownTable +} + +// Ancients returns the last of the frozen items. +func (f *prunedfreezer) Ancients() (uint64, error) { + return atomic.LoadUint64(&f.frozen), nil +} + +// ItemAmountInAncient returns the actual length of current ancientDB, return 0. +func (f *prunedfreezer) ItemAmountInAncient() (uint64, error) { + return 0, nil +} + +// AncientOffSet returns the offset of current ancientDB, offset == frozen. +func (f *prunedfreezer) AncientOffSet() uint64 { + return atomic.LoadUint64(&f.frozen) +} + +// AncientSize returns the ancient size of the specified category, return 0. +func (f *prunedfreezer) AncientSize(kind string) (uint64, error) { + if _, ok := FreezerNoSnappy[kind]; ok { + return 0, nil + } + return 0, errUnknownTable +} + +// AppendAncient update frozen. +// +// Notably, this function is lock free but kind of thread-safe. All out-of-order +// injection will be rejected. But if two injections with same number happen at +// the same time, we can get into the trouble. +func (f *prunedfreezer) AppendAncient(number uint64, hash, header, body, receipts, td []byte) (err error) { + if atomic.LoadUint64(&f.frozen) != number { + return errOutOrderInsertion + } + atomic.AddUint64(&f.frozen, 1) + return nil +} + +// TruncateAncients discards any recent data above the provided threshold number, always success. +func (f *prunedfreezer) TruncateAncients(items uint64) error { + if atomic.LoadUint64(&f.frozen) <= items { + return nil + } + atomic.StoreUint64(&f.frozen, items) + WriteFrozenOfAncientFreezer(f.db, atomic.LoadUint64(&f.frozen)) + return nil +} + +// Sync flushes meta data tables to disk. +func (f *prunedfreezer) Sync() error { + WriteFrozenOfAncientFreezer(f.db, atomic.LoadUint64(&f.frozen)) + return nil +} + +// freeze is a background thread that periodically checks the blockchain for any +// import progress and moves ancient data from the fast database into the freezer. +// +// This functionality is deliberately broken off from block importing to avoid +// incurring additional data shuffling delays on block propagation. +func (f *prunedfreezer) freeze() { + nfdb := &nofreezedb{KeyValueStore: f.db} + + var backoff bool + for { + select { + case <-f.quit: + log.Info("Freezer shutting down") + return + default: + } + if backoff { + select { + case <-time.NewTimer(freezerRecheckInterval).C: + case <-f.quit: + return + } + } + + // Retrieve the freezing threshold. + hash := ReadHeadBlockHash(nfdb) + if hash == (common.Hash{}) { + log.Debug("Current full block hash unavailable") // new chain, empty database + backoff = true + continue + } + number := ReadHeaderNumber(nfdb, hash) + threshold := atomic.LoadUint64(&f.threshold) + + switch { + case number == nil: + log.Error("Current full block number unavailable", "hash", hash) + backoff = true + continue + + case *number < threshold: + log.Debug("Current full block not old enough", "number", *number, "hash", hash, "delay", threshold) + backoff = true + continue + + case *number-threshold <= f.frozen: + log.Debug("Ancient blocks frozen already", "number", *number, "hash", hash, "frozen", f.frozen) + backoff = true + continue + } + head := ReadHeader(nfdb, hash, *number) + if head == nil { + log.Error("Stable state block unavailable", "number", *number, "hash", hash) + backoff = true + continue + } + + stableStabeNumber := ReadSafePointBlockNumber(nfdb) + switch { + case stableStabeNumber < params.StableStateThreshold: + log.Debug("Stable state block not old enough", "number", stableStabeNumber) + backoff = true + continue + + case stableStabeNumber > *number: + log.Warn("Stable state block biger current full block", "number", stableStabeNumber, "number", *number) + backoff = true + continue + } + stableStabeNumber -= params.StableStateThreshold + + // Seems we have data ready to be frozen, process in usable batches + limit := *number - threshold + if limit > stableStabeNumber { + limit = stableStabeNumber + } + + if limit < f.frozen { + log.Debug("Stable state block has prune", "limit", limit, "frozen", f.frozen) + backoff = true + continue + } + + if limit-f.frozen > freezerBatchLimit { + limit = f.frozen + freezerBatchLimit + } + var ( + start = time.Now() + first = f.frozen + ancients = make([]common.Hash, 0, limit-f.frozen) + ) + for f.frozen <= limit { + // Retrieves all the components of the canonical block + hash := ReadCanonicalHash(nfdb, f.frozen) + if hash == (common.Hash{}) { + log.Error("Canonical hash missing, can't freeze", "number", f.frozen) + } + log.Trace("Deep froze ancient block", "number", f.frozen, "hash", hash) + // Inject all the components into the relevant data tables + if err := f.AppendAncient(f.frozen, nil, nil, nil, nil, nil); err != nil { + log.Error("Append ancient err", "number", f.frozen, "hash", hash, "err", err) + break + } + if hash != (common.Hash{}) { + ancients = append(ancients, hash) + } + } + // Batch of blocks have been frozen, flush them before wiping from leveldb + if err := f.Sync(); err != nil { + log.Crit("Failed to flush frozen tables", "err", err) + } + backoff = f.frozen-first >= freezerBatchLimit + gcKvStore(f.db, ancients, first, f.frozen, start) + } +} + +func (f *prunedfreezer) ReadAncients(fn func(ethdb.AncientReader) error) (err error) { + return fn(f) +} + +func (f *prunedfreezer) AncientRange(kind string, start, count, maxBytes uint64) ([][]byte, error) { + return nil, errNotSupported +} + +func (f *prunedfreezer) ModifyAncients(func(ethdb.AncientWriteOp) error) (int64, error) { + return 0, errNotSupported +} diff --git a/core/rawdb/schema.go b/core/rawdb/schema.go index b64b570356..e04f94e7d4 100644 --- a/core/rawdb/schema.go +++ b/core/rawdb/schema.go @@ -75,6 +75,15 @@ var ( //offSet of the ancientDB before updated version. offSetOfLastAncientFreezer = []byte("offSetOfLastAncientFreezer") + //frozenOfAncientDBKey tracks the block number for ancientDB to save. + frozenOfAncientDBKey = []byte("FrozenOfAncientDB") + + //LastSafePointBlockKey tracks the block number for block state that write disk + LastSafePointBlockKey = []byte("LastSafePointBlockNumber") + + //PruneAncientFlag flag whether prune ancient + pruneAncientKey = []byte("PruneAncientFlag") + // badBlockKey tracks the list of bad blocks seen by local badBlockKey = []byte("InvalidBlock") diff --git a/core/remote_state_verifier.go b/core/remote_state_verifier.go new file mode 100644 index 0000000000..cb8d2366bb --- /dev/null +++ b/core/remote_state_verifier.go @@ -0,0 +1,446 @@ +package core + +import ( + "fmt" + "math/big" + "math/rand" + "sync" + "time" + + lru "github.com/hashicorp/golang-lru" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/event" + "github.com/ethereum/go-ethereum/log" + "github.com/ethereum/go-ethereum/metrics" +) + +const ( + verifiedCacheSize = 256 + maxForkHeight = 11 + + // defaultPeerNumber is default number of verify peers + defaultPeerNumber = 3 + // pruneHeightDiff indicates that if the height difference between current block and task's + // corresponding block is larger than it, the task should be pruned. + pruneHeightDiff = 15 + pruneInterval = 5 * time.Second + resendInterval = 2 * time.Second + // tryAllPeersTime is the time that a block has not been verified and then try all the valid verify peers. + tryAllPeersTime = 15 * time.Second + // maxWaitVerifyResultTime is the max time of waiting for ancestor's verify result. + maxWaitVerifyResultTime = 30 * time.Second +) + +var ( + verifyTaskCounter = metrics.NewRegisteredCounter("verifymanager/task/total", nil) + verifyTaskSucceedMeter = metrics.NewRegisteredMeter("verifymanager/task/result/succeed", nil) + verifyTaskFailedMeter = metrics.NewRegisteredMeter("verifymanager/task/result/failed", nil) + + verifyTaskExecutionTimer = metrics.NewRegisteredTimer("verifymanager/task/execution", nil) +) + +type remoteVerifyManager struct { + bc *BlockChain + taskLock sync.RWMutex + tasks map[common.Hash]*verifyTask + peers verifyPeers + verifiedCache *lru.Cache + allowInsecure bool + + // Subscription + chainBlockCh chan ChainHeadEvent + chainHeadSub event.Subscription + + // Channels + verifyCh chan common.Hash + messageCh chan verifyMessage +} + +func NewVerifyManager(blockchain *BlockChain, peers verifyPeers, allowInsecure bool) (*remoteVerifyManager, error) { + verifiedCache, _ := lru.New(verifiedCacheSize) + block := blockchain.CurrentBlock() + if block == nil { + return nil, ErrCurrentBlockNotFound + } + + // rewind to last non verified block + number := new(big.Int).Sub(block.Number(), big.NewInt(int64(maxForkHeight))) + if number.Cmp(common.Big0) < 0 { + blockchain.SetHead(0) + } else { + numberU64 := number.Uint64() + blockchain.SetHead(numberU64) + block := blockchain.GetBlockByNumber(numberU64) + for i := 0; i < maxForkHeight && block.NumberU64() > 0; i++ { + // When inserting a block, + // the block before 11 blocks will be verified, + // so the parent block of 11-22 will directly write the verification information. + verifiedCache.Add(block.Hash(), true) + block = blockchain.GetBlockByHash(block.ParentHash()) + if block == nil { + return nil, fmt.Errorf("block is nil, number: %d", number) + } + } + } + + vm := &remoteVerifyManager{ + bc: blockchain, + tasks: make(map[common.Hash]*verifyTask), + peers: peers, + verifiedCache: verifiedCache, + allowInsecure: allowInsecure, + + chainBlockCh: make(chan ChainHeadEvent, chainHeadChanSize), + verifyCh: make(chan common.Hash, maxForkHeight), + messageCh: make(chan verifyMessage), + } + vm.chainHeadSub = blockchain.SubscribeChainBlockEvent(vm.chainBlockCh) + return vm, nil +} + +func (vm *remoteVerifyManager) mainLoop() { + defer vm.chainHeadSub.Unsubscribe() + + pruneTicker := time.NewTicker(pruneInterval) + defer pruneTicker.Stop() + for { + select { + case h := <-vm.chainBlockCh: + vm.NewBlockVerifyTask(h.Block.Header()) + case hash := <-vm.verifyCh: + vm.cacheBlockVerified(hash) + vm.taskLock.Lock() + if task, ok := vm.tasks[hash]; ok { + vm.CloseTask(task) + verifyTaskSucceedMeter.Mark(1) + verifyTaskExecutionTimer.Update(time.Since(task.startAt)) + } + vm.taskLock.Unlock() + case <-pruneTicker.C: + vm.taskLock.Lock() + for _, task := range vm.tasks { + if vm.bc.insertStopped() || (vm.bc.CurrentHeader().Number.Cmp(task.blockHeader.Number) == 1 && + vm.bc.CurrentHeader().Number.Uint64()-task.blockHeader.Number.Uint64() > pruneHeightDiff) { + vm.CloseTask(task) + verifyTaskFailedMeter.Mark(1) + } + } + vm.taskLock.Unlock() + case message := <-vm.messageCh: + vm.taskLock.RLock() + if vt, ok := vm.tasks[message.verifyResult.BlockHash]; ok { + vt.messageCh <- message + } + vm.taskLock.RUnlock() + // System stopped + case <-vm.bc.quit: + vm.taskLock.RLock() + for _, task := range vm.tasks { + task.Close() + } + vm.taskLock.RUnlock() + return + case <-vm.chainHeadSub.Err(): + return + } + } +} + +func (vm *remoteVerifyManager) NewBlockVerifyTask(header *types.Header) { + for i := 0; header != nil && i <= maxForkHeight; i++ { + // if is genesis block, mark it as verified and break. + if header.Number.Uint64() == 0 { + vm.cacheBlockVerified(header.Hash()) + break + } + func(hash common.Hash) { + // if verified cache record that this block has been verified, skip. + if _, ok := vm.verifiedCache.Get(hash); ok { + return + } + // if there already has a verify task for this block, skip. + vm.taskLock.RLock() + _, ok := vm.tasks[hash] + vm.taskLock.RUnlock() + if ok { + return + } + + if header.TxHash == types.EmptyRootHash { + log.Debug("this is an empty block:", "block", hash, "number", header.Number) + vm.cacheBlockVerified(hash) + return + } + + var diffLayer *types.DiffLayer + if cached, ok := vm.bc.diffLayerChanCache.Get(hash); ok { + diffLayerCh := cached.(chan struct{}) + <-diffLayerCh + diffLayer = vm.bc.GetTrustedDiffLayer(hash) + } + // if this block has no diff, there is no need to verify it. + if diffLayer == nil { + log.Info("block's trusted diffLayer is nil", "hash", hash, "number", header.Number) + return + } + diffHash, err := CalculateDiffHash(diffLayer) + if err != nil { + log.Error("failed to get diff hash", "block", hash, "number", header.Number, "error", err) + return + } + verifyTask := NewVerifyTask(diffHash, header, vm.peers, vm.verifyCh, vm.allowInsecure) + vm.taskLock.Lock() + vm.tasks[hash] = verifyTask + vm.taskLock.Unlock() + verifyTaskCounter.Inc(1) + }(header.Hash()) + header = vm.bc.GetHeaderByHash(header.ParentHash) + } +} + +func (vm *remoteVerifyManager) cacheBlockVerified(hash common.Hash) { + if vm.verifiedCache.Len() >= verifiedCacheSize { + vm.verifiedCache.RemoveOldest() + } + vm.verifiedCache.Add(hash, true) +} + +// AncestorVerified function check block has been verified or it's a empty block. +func (vm *remoteVerifyManager) AncestorVerified(header *types.Header) bool { + // find header of H-11 block. + header = vm.bc.GetHeaderByNumber(header.Number.Uint64() - maxForkHeight) + // If start from genesis block, there has not a H-11 block. + if header == nil { + return true + } + + hash := header.Hash() + + // Check if the task is complete + vm.taskLock.RLock() + task, exist := vm.tasks[hash] + vm.taskLock.RUnlock() + timeout := time.NewTimer(maxWaitVerifyResultTime) + defer timeout.Stop() + if exist { + select { + case <-task.terminalCh: + case <-timeout.C: + return false + } + } + + _, exist = vm.verifiedCache.Get(hash) + return exist +} + +func (vm *remoteVerifyManager) HandleRootResponse(vr *VerifyResult, pid string) error { + vm.messageCh <- verifyMessage{verifyResult: vr, peerId: pid} + return nil +} + +func (vm *remoteVerifyManager) CloseTask(task *verifyTask) { + delete(vm.tasks, task.blockHeader.Hash()) + task.Close() + verifyTaskCounter.Dec(1) +} + +type VerifyResult struct { + Status types.VerifyStatus + BlockNumber uint64 + BlockHash common.Hash + Root common.Hash +} + +type verifyMessage struct { + verifyResult *VerifyResult + peerId string +} + +type verifyTask struct { + diffhash common.Hash + blockHeader *types.Header + candidatePeers verifyPeers + badPeers map[string]struct{} + startAt time.Time + allowInsecure bool + + messageCh chan verifyMessage + terminalCh chan struct{} +} + +func NewVerifyTask(diffhash common.Hash, header *types.Header, peers verifyPeers, verifyCh chan common.Hash, allowInsecure bool) *verifyTask { + vt := &verifyTask{ + diffhash: diffhash, + blockHeader: header, + candidatePeers: peers, + badPeers: make(map[string]struct{}), + allowInsecure: allowInsecure, + messageCh: make(chan verifyMessage), + terminalCh: make(chan struct{}), + } + go vt.Start(verifyCh) + return vt +} + +func (vt *verifyTask) Close() { + // It is safe to call close multiple + select { + case <-vt.terminalCh: + default: + close(vt.terminalCh) + } +} + +func (vt *verifyTask) Start(verifyCh chan common.Hash) { + vt.startAt = time.Now() + + vt.sendVerifyRequest(defaultPeerNumber) + resend := time.NewTicker(resendInterval) + defer resend.Stop() + for { + select { + case msg := <-vt.messageCh: + switch msg.verifyResult.Status { + case types.StatusFullVerified: + vt.compareRootHashAndMark(msg, verifyCh) + case types.StatusPartiallyVerified: + log.Warn("block is insecure verified", "hash", msg.verifyResult.BlockHash, "number", msg.verifyResult.BlockNumber) + if vt.allowInsecure { + vt.compareRootHashAndMark(msg, verifyCh) + } + case types.StatusDiffHashMismatch, types.StatusImpossibleFork, types.StatusUnexpectedError: + vt.badPeers[msg.peerId] = struct{}{} + log.Info("peer is not available", "hash", msg.verifyResult.BlockHash, "number", msg.verifyResult.BlockNumber, "peer", msg.peerId, "reason", msg.verifyResult.Status.Msg) + case types.StatusBlockTooNew, types.StatusBlockNewer, types.StatusPossibleFork: + log.Info("return msg from peer", "peerId", msg.peerId, "hash", msg.verifyResult.BlockHash, "msg", msg.verifyResult.Status.Msg) + } + newVerifyMsgTypeGauge(msg.verifyResult.Status.Code, msg.peerId).Inc(1) + case <-resend.C: + // if a task has run over 15s, try all the vaild peers to verify. + if time.Since(vt.startAt) < tryAllPeersTime { + vt.sendVerifyRequest(1) + } else { + vt.sendVerifyRequest(-1) + } + case <-vt.terminalCh: + return + } + } +} + +// sendVerifyRequest func select at most n peers from (candidatePeers-badPeers) randomly and send verify request. +// when n<0, send to all the peers exclude badPeers. +func (vt *verifyTask) sendVerifyRequest(n int) { + var validPeers []VerifyPeer + candidatePeers := vt.candidatePeers.GetVerifyPeers() + for _, p := range candidatePeers { + if _, ok := vt.badPeers[p.ID()]; !ok { + validPeers = append(validPeers, p) + } + } + // if has not valid peer, log warning. + if len(validPeers) == 0 { + log.Warn("there is no valid peer for block", "number", vt.blockHeader.Number) + return + } + + if n < len(validPeers) && n > 0 { + rand.Seed(time.Now().UnixNano()) + rand.Shuffle(len(validPeers), func(i, j int) { validPeers[i], validPeers[j] = validPeers[j], validPeers[i] }) + } else { + n = len(validPeers) + } + for i := 0; i < n; i++ { + p := validPeers[i] + p.RequestRoot(vt.blockHeader.Number.Uint64(), vt.blockHeader.Hash(), vt.diffhash) + } +} + +func (vt *verifyTask) compareRootHashAndMark(msg verifyMessage, verifyCh chan common.Hash) { + if msg.verifyResult.Root == vt.blockHeader.Root { + // write back to manager so that manager can cache the result and delete this task. + verifyCh <- msg.verifyResult.BlockHash + } else { + vt.badPeers[msg.peerId] = struct{}{} + } +} + +type VerifyPeer interface { + RequestRoot(blockNumber uint64, blockHash common.Hash, diffHash common.Hash) error + ID() string +} + +type verifyPeers interface { + GetVerifyPeers() []VerifyPeer +} + +type VerifyMode uint32 + +const ( + LocalVerify VerifyMode = iota + FullVerify + InsecureVerify + NoneVerify +) + +func (mode VerifyMode) IsValid() bool { + return mode >= LocalVerify && mode <= NoneVerify +} + +func (mode VerifyMode) String() string { + switch mode { + case LocalVerify: + return "local" + case FullVerify: + return "full" + case InsecureVerify: + return "insecure" + case NoneVerify: + return "none" + default: + return "unknown" + } +} + +func (mode VerifyMode) MarshalText() ([]byte, error) { + switch mode { + case LocalVerify: + return []byte("local"), nil + case FullVerify: + return []byte("full"), nil + case InsecureVerify: + return []byte("insecure"), nil + case NoneVerify: + return []byte("none"), nil + default: + return nil, fmt.Errorf("unknown verify mode %d", mode) + } +} + +func (mode *VerifyMode) UnmarshalText(text []byte) error { + switch string(text) { + case "local": + *mode = LocalVerify + case "full": + *mode = FullVerify + case "insecure": + *mode = InsecureVerify + case "none": + *mode = NoneVerify + default: + return fmt.Errorf(`unknown sync mode %q, want "full", "light" or "insecure"`, text) + } + return nil +} + +func (mode VerifyMode) NeedRemoteVerify() bool { + return mode == FullVerify || mode == InsecureVerify +} + +func newVerifyMsgTypeGauge(msgType uint16, peerId string) metrics.Gauge { + m := fmt.Sprintf("verifymanager/message/%d/peer/%s", msgType, peerId) + return metrics.GetOrRegisterGauge(m, nil) +} diff --git a/core/state/database.go b/core/state/database.go index 350f513948..0f31bc9139 100644 --- a/core/state/database.go +++ b/core/state/database.go @@ -75,6 +75,9 @@ type Database interface { // Purge cache Purge() + + // NoTries returns whether the database has tries storage. + NoTries() bool } // Trie is a Ethereum Merkle Patricia trie. @@ -138,10 +141,12 @@ func NewDatabase(db ethdb.Database) Database { func NewDatabaseWithConfig(db ethdb.Database, config *trie.Config) Database { csc, _ := lru.New(codeSizeCacheSize) cc, _ := lru.New(codeCacheSize) + noTries := config != nil && config.NoTries return &cachingDB{ db: trie.NewDatabaseWithConfig(db, config), codeSizeCache: csc, codeCache: cc, + noTries: noTries, } } @@ -150,6 +155,7 @@ func NewDatabaseWithConfigAndCache(db ethdb.Database, config *trie.Config) Datab cc, _ := lru.New(codeCacheSize) atc, _ := lru.New(accountTrieCacheSize) stc, _ := lru.New(storageTrieCacheSize) + noTries := config != nil && config.NoTries database := &cachingDB{ db: trie.NewDatabaseWithConfig(db, config), @@ -157,8 +163,11 @@ func NewDatabaseWithConfigAndCache(db ethdb.Database, config *trie.Config) Datab codeCache: cc, accountTrieCache: atc, storageTrieCache: stc, + noTries: noTries, + } + if !noTries { + go database.purgeLoop() } - go database.purgeLoop() return database } @@ -168,6 +177,7 @@ type cachingDB struct { codeCache *lru.Cache accountTrieCache *lru.Cache storageTrieCache *lru.Cache + noTries bool } type triePair struct { @@ -191,6 +201,9 @@ func (db *cachingDB) purgeLoop() { // OpenTrie opens the main account trie at a specific root hash. func (db *cachingDB) OpenTrie(root common.Hash) (Trie, error) { + if db.noTries { + return trie.NewEmptyTrie(), nil + } if db.accountTrieCache != nil { if tr, exist := db.accountTrieCache.Get(root); exist { return tr.(Trie).(*trie.SecureTrie).Copy(), nil @@ -205,6 +218,9 @@ func (db *cachingDB) OpenTrie(root common.Hash) (Trie, error) { // OpenStorageTrie opens the storage trie of an account. func (db *cachingDB) OpenStorageTrie(addrHash, root common.Hash) (Trie, error) { + if db.noTries { + return trie.NewEmptyTrie(), nil + } if db.storageTrieCache != nil { if tries, exist := db.storageTrieCache.Get(addrHash); exist { triesPairs := tries.([3]*triePair) @@ -250,6 +266,10 @@ func (db *cachingDB) CacheStorage(addrHash common.Hash, root common.Hash, t Trie } } +func (db *cachingDB) NoTries() bool { + return db.noTries +} + func (db *cachingDB) Purge() { if db.storageTrieCache != nil { db.storageTrieCache.Purge() @@ -267,6 +287,8 @@ func (db *cachingDB) CopyTrie(t Trie) Trie { switch t := t.(type) { case *trie.SecureTrie: return t.Copy() + case *trie.EmptyTrie: + return t.Copy() default: panic(fmt.Errorf("unknown trie type %T", t)) } diff --git a/core/state/pruner/pruner.go b/core/state/pruner/pruner.go index 459f8ecc1d..3dd167da49 100644 --- a/core/state/pruner/pruner.go +++ b/core/state/pruner/pruner.go @@ -31,7 +31,9 @@ import ( "github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/consensus" + "github.com/ethereum/go-ethereum/core" "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/core/state" "github.com/ethereum/go-ethereum/core/state/snapshot" "github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/crypto" @@ -102,7 +104,7 @@ func NewPruner(db ethdb.Database, datadir, trieCachePath string, bloomSize, trie if headBlock == nil { return nil, errors.New("Failed to load head block") } - snaptree, err := snapshot.New(db, trie.NewDatabase(db), 256, int(triesInMemory), headBlock.Root(), false, false, false) + snaptree, err := snapshot.New(db, trie.NewDatabase(db), 256, int(triesInMemory), headBlock.Root(), false, false, false, false) if err != nil { return nil, err // The relevant snapshot(s) might not exist } @@ -137,6 +139,105 @@ func NewBlockPruner(db ethdb.Database, n *node.Node, oldAncientPath, newAncientP } } +func NewAllPruner(db ethdb.Database) (*Pruner, error) { + headBlock := rawdb.ReadHeadBlock(db) + if headBlock == nil { + return nil, errors.New("Failed to load head block") + } + return &Pruner{ + db: db, + }, nil +} + +func (p *Pruner) PruneAll(genesis *core.Genesis) error { + deleteCleanTrieCache(p.trieCachePath) + return pruneAll(p.db, genesis) +} + +func pruneAll(maindb ethdb.Database, g *core.Genesis) error { + var ( + count int + size common.StorageSize + pstart = time.Now() + logged = time.Now() + batch = maindb.NewBatch() + iter = maindb.NewIterator(nil, nil) + ) + start := time.Now() + for iter.Next() { + key := iter.Key() + if len(key) == common.HashLength { + count += 1 + size += common.StorageSize(len(key) + len(iter.Value())) + batch.Delete(key) + + var eta time.Duration // Realistically will never remain uninited + if done := binary.BigEndian.Uint64(key[:8]); done > 0 { + var ( + left = math.MaxUint64 - binary.BigEndian.Uint64(key[:8]) + speed = done/uint64(time.Since(pstart)/time.Millisecond+1) + 1 // +1s to avoid division by zero + ) + eta = time.Duration(left/speed) * time.Millisecond + } + if time.Since(logged) > 8*time.Second { + log.Info("Pruning state data", "nodes", count, "size", size, + "elapsed", common.PrettyDuration(time.Since(pstart)), "eta", common.PrettyDuration(eta)) + logged = time.Now() + } + // Recreate the iterator after every batch commit in order + // to allow the underlying compactor to delete the entries. + if batch.ValueSize() >= ethdb.IdealBatchSize { + batch.Write() + batch.Reset() + + iter.Release() + iter = maindb.NewIterator(nil, key) + } + } + } + if batch.ValueSize() > 0 { + batch.Write() + batch.Reset() + } + iter.Release() + log.Info("Pruned state data", "nodes", count, "size", size, "elapsed", common.PrettyDuration(time.Since(pstart))) + + // Start compactions, will remove the deleted data from the disk immediately. + // Note for small pruning, the compaction is skipped. + if count >= rangeCompactionThreshold { + cstart := time.Now() + for b := 0x00; b <= 0xf0; b += 0x10 { + var ( + start = []byte{byte(b)} + end = []byte{byte(b + 0x10)} + ) + if b == 0xf0 { + end = nil + } + log.Info("Compacting database", "range", fmt.Sprintf("%#x-%#x", start, end), "elapsed", common.PrettyDuration(time.Since(cstart))) + if err := maindb.Compact(start, end); err != nil { + log.Error("Database compaction failed", "error", err) + return err + } + } + log.Info("Database compaction finished", "elapsed", common.PrettyDuration(time.Since(cstart))) + } + statedb, _ := state.New(common.Hash{}, state.NewDatabase(maindb), nil) + for addr, account := range g.Alloc { + statedb.AddBalance(addr, account.Balance) + statedb.SetCode(addr, account.Code) + statedb.SetNonce(addr, account.Nonce) + for key, value := range account.Storage { + statedb.SetState(addr, key, value) + } + } + root := statedb.IntermediateRoot(false) + statedb.Commit(nil) + statedb.Database().TrieDB().Commit(root, true, nil) + log.Info("State pruning successful", "pruned", size, "elapsed", common.PrettyDuration(time.Since(start))) + return nil +} + func prune(snaptree *snapshot.Tree, root common.Hash, maindb ethdb.Database, stateBloom *stateBloom, bloomPath string, middleStateRoots map[common.Hash]struct{}, start time.Time) error { // Delete all stale trie nodes in the disk. With the help of state bloom // the trie nodes(and codes) belong to the active state will be filtered @@ -257,7 +358,7 @@ func prune(snaptree *snapshot.Tree, root common.Hash, maindb ethdb.Database, sta func (p *BlockPruner) backUpOldDb(name string, cache, handles int, namespace string, readonly, interrupt bool) error { // Open old db wrapper. - chainDb, err := p.node.OpenDatabaseWithFreezer(name, cache, handles, p.oldAncientPath, namespace, readonly, true, interrupt) + chainDb, err := p.node.OpenDatabaseWithFreezer(name, cache, handles, p.oldAncientPath, namespace, readonly, true, interrupt, false) if err != nil { log.Error("Failed to open ancient database", "err=", err) return err @@ -584,7 +685,7 @@ func RecoverPruning(datadir string, db ethdb.Database, trieCachePath string, tri // - The state HEAD is rewound already because of multiple incomplete `prune-state` // In this case, even the state HEAD is not exactly matched with snapshot, it // still feasible to recover the pruning correctly. - snaptree, err := snapshot.New(db, trie.NewDatabase(db), 256, int(triesInMemory), headBlock.Root(), false, false, true) + snaptree, err := snapshot.New(db, trie.NewDatabase(db), 256, int(triesInMemory), headBlock.Root(), false, false, true, false) if err != nil { return err // The relevant snapshot(s) might not exist } diff --git a/core/state/snapshot/journal.go b/core/state/snapshot/journal.go index 99b152b29b..883d947c90 100644 --- a/core/state/snapshot/journal.go +++ b/core/state/snapshot/journal.go @@ -149,7 +149,7 @@ func loadAndParseJournal(db ethdb.KeyValueStore, base *diskLayer) (snapshot, jou } // loadSnapshot loads a pre-existing state snapshot backed by a key-value store. -func loadSnapshot(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache int, root common.Hash, recovery bool) (snapshot, bool, error) { +func loadSnapshot(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache int, root common.Hash, recovery, withoutTrie bool) (snapshot, bool, error) { // If snapshotting is disabled (initial sync in progress), don't do anything, // wait for the chain to permit us to do something meaningful if rawdb.ReadSnapshotDisabled(diskdb) { @@ -168,6 +168,7 @@ func loadSnapshot(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache int, root: baseRoot, } snapshot, generator, err := loadAndParseJournal(diskdb, base) + if err != nil { log.Warn("Failed to load new-format journal", "error", err) return nil, false, err @@ -181,6 +182,11 @@ func loadSnapshot(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache int, // which is below the snapshot. In this case the snapshot can be recovered // by re-executing blocks but right now it's unavailable. if head := snapshot.Root(); head != root { + log.Warn("Snapshot is not continuous with chain", "snaproot", head, "chainroot", root) + + if withoutTrie { + return snapshot, false, nil + } // If it's legacy snapshot, or it's new-format snapshot but // it's not in recovery mode, returns the error here for // rebuilding the entire snapshot forcibly. @@ -191,7 +197,6 @@ func loadSnapshot(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache int, // the disk layer is always higher than chain head. It can // be eventually recovered when the chain head beyonds the // disk layer. - log.Warn("Snapshot is not continuous with chain", "snaproot", head, "chainroot", root) } // Everything loaded correctly, resume any suspended operations if !generator.Done { diff --git a/core/state/snapshot/snapshot.go b/core/state/snapshot/snapshot.go index b1ffac82ba..967fc0c68c 100644 --- a/core/state/snapshot/snapshot.go +++ b/core/state/snapshot/snapshot.go @@ -192,18 +192,12 @@ type Tree struct { // store (with a number of memory layers from a journal), ensuring that the head // of the snapshot matches the expected one. // -// If the snapshot is missing or the disk layer is broken, the snapshot will be -// reconstructed using both the existing data and the state trie. -// The repair happens on a background thread. -// -// If the memory layers in the journal do not match the disk layer (e.g. there is -// a gap) or the journal is missing, there are two repair cases: -// -// - if the 'recovery' parameter is true, all memory diff-layers will be discarded. -// This case happens when the snapshot is 'ahead' of the state trie. -// - otherwise, the entire snapshot is considered invalid and will be recreated on -// a background thread. -func New(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache, cap int, root common.Hash, async bool, rebuild bool, recovery bool) (*Tree, error) { +// If the snapshot is missing or the disk layer is broken, the entire is deleted +// and will be reconstructed from scratch based on the tries in the key-value +// store, on a background thread. If the memory layers from the journal is not +// continuous with disk layer or the journal is missing, all diffs will be discarded +// iff it's in "recovery" mode, otherwise rebuild is mandatory. +func New(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache, cap int, root common.Hash, async bool, rebuild bool, recovery, withoutTrie bool) (*Tree, error) { // Create a new, empty snapshot tree snap := &Tree{ diskdb: diskdb, @@ -216,7 +210,7 @@ func New(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache, cap int, root defer snap.waitBuild() } // Attempt to load a previously persisted snapshot and rebuild one if failed - head, disabled, err := loadSnapshot(diskdb, triedb, cache, root, recovery) + head, disabled, err := loadSnapshot(diskdb, triedb, cache, root, recovery, withoutTrie) if disabled { log.Warn("Snapshot maintenance disabled (syncing)") return snap, nil diff --git a/core/state/state_object.go b/core/state/state_object.go index dc57fc67ba..780cd6d255 100644 --- a/core/state/state_object.go +++ b/core/state/state_object.go @@ -18,6 +18,7 @@ package state import ( "bytes" + "errors" "fmt" "io" "math/big" @@ -25,12 +26,15 @@ import ( "time" "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core/state/snapshot" "github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/crypto" "github.com/ethereum/go-ethereum/metrics" "github.com/ethereum/go-ethereum/rlp" ) +const snapshotStaleRetryInterval = time.Millisecond * 10 + var emptyCodeHash = crypto.Keccak256(nil) type Code []byte @@ -267,7 +271,19 @@ func (s *StateObject) GetCommittedState(db Database, key common.Hash) common.Has } enc, err = s.db.snap.Storage(s.addrHash, crypto.Keccak256Hash(key.Bytes())) } - // If the snapshot is unavailable or reading from it fails, load from the database. + // ErrSnapshotStale may occur due to diff layers in the update, so we should try again in noTrie mode. + if s.db.NoTrie() && err != nil && errors.Is(err, snapshot.ErrSnapshotStale) { + // This error occurs when statedb.snaps.Cap changes the state of the merged difflayer + // to stale during the refresh of the difflayer, indicating that it is about to be discarded. + // Since the difflayer is refreshed in parallel, + // there is a small chance that the difflayer of the stale will be read while reading, + // resulting in an empty array being returned here. + // Therefore, noTrie mode must retry here, + // and add a time interval when retrying to avoid stacking too much and causing stack overflow. + time.Sleep(snapshotStaleRetryInterval) + return s.GetCommittedState(db, key) + } + // If snapshot unavailable or reading from it failed, load from the database if s.db.snap == nil || err != nil { if meter != nil { // If we already spent time checking the snapshot, account for it @@ -383,12 +399,11 @@ func (s *StateObject) updateTrie(db Database) Trie { s.db.MetricsMux.Unlock() }(time.Now()) } - // The snapshot storage map for the object - var storage map[string][]byte // Insert all the pending updates into the trie tr := s.getTrie(db) usedStorage := make([][]byte, 0, len(s.pendingStorage)) + dirtyStorage := make(map[common.Hash][]byte) for key, value := range s.pendingStorage { // Skip noop changes, persist actual changes if value == s.originStorage[key] { @@ -405,21 +420,41 @@ func (s *StateObject) updateTrie(db Database) Trie { s.setError(tr.TryUpdate(key[:], v)) s.db.StorageUpdated += 1 } + dirtyStorage[key] = v + } + var wg sync.WaitGroup + wg.Add(1) + go func() { + defer wg.Done() + for key, value := range dirtyStorage { + if len(value) == 0 { + s.setError(tr.TryDelete(key[:])) + } else { + s.setError(tr.TryUpdate(key[:], value)) + } + usedStorage = append(usedStorage, common.CopyBytes(key[:])) + } + }() + if s.db.snap != nil { // If state snapshotting is active, cache the data til commit - if s.db.snap != nil { - s.db.snapMux.Lock() + wg.Add(1) + go func() { + defer wg.Done() + s.db.snapStorageMux.Lock() + // The snapshot storage map for the object + storage := s.db.snapStorage[s.address] if storage == nil { - // Retrieve the old storage map, if available, create a new one otherwise - if storage = s.db.snapStorage[s.address]; storage == nil { - storage = make(map[string][]byte) - s.db.snapStorage[s.address] = storage - } + storage = make(map[string][]byte, len(dirtyStorage)) + s.db.snapStorage[s.address] = storage } - storage[string(key[:])] = v // v will be nil if value is 0x00 - s.db.snapMux.Unlock() - } - usedStorage = append(usedStorage, common.CopyBytes(key[:])) // Copy needed for closure + for key, value := range dirtyStorage { + storage[string(key[:])] = value + } + s.db.snapStorageMux.Unlock() + }() } + wg.Wait() + if s.db.prefetcher != nil { s.db.prefetcher.used(s.data.Root, usedStorage) } @@ -431,6 +466,13 @@ func (s *StateObject) updateTrie(db Database) Trie { // UpdateRoot sets the trie root to the current root hash of func (s *StateObject) updateRoot(db Database) { + // If node runs in no trie mode, set root to empty. + defer func() { + if db.NoTries() { + s.data.Root = common.Hash{} + } + }() + // If nothing changed, don't bother with hashing anything if s.updateTrie(db) == nil { return diff --git a/core/state/statedb.go b/core/state/statedb.go index ba7a238e72..43e99e3f14 100644 --- a/core/state/statedb.go +++ b/core/state/statedb.go @@ -82,6 +82,7 @@ type StateDB struct { stateRoot common.Hash // The calculation result of IntermediateRoot trie Trie + noTrie bool hasher crypto.KeccakState diffLayer *types.DiffLayer diffTries map[common.Address]Trie @@ -90,12 +91,13 @@ type StateDB struct { fullProcessed bool pipeCommit bool - snapMux sync.Mutex - snaps *snapshot.Tree - snap snapshot.Snapshot - snapDestructs map[common.Address]struct{} - snapAccounts map[common.Address][]byte - snapStorage map[common.Address]map[string][]byte + snaps *snapshot.Tree + snap snapshot.Snapshot + snapAccountMux sync.Mutex // Mutex for snap account access + snapStorageMux sync.Mutex // Mutex for snap storage access + snapDestructs map[common.Address]struct{} + snapAccounts map[common.Address][]byte + snapStorage map[common.Address]map[string][]byte // This map holds 'live' objects, which will get modified while processing a state transition. stateObjects map[common.Address]*StateObject @@ -178,6 +180,7 @@ func newStateDB(root common.Hash, db Database, snaps *snapshot.Tree) (*StateDB, journal: newJournal(), hasher: crypto.NewKeccakState(), } + if sdb.snaps != nil { if sdb.snap = sdb.snaps.Snapshot(root); sdb.snap != nil { sdb.snapDestructs = make(map[common.Address]struct{}) @@ -192,6 +195,7 @@ func newStateDB(root common.Hash, db Database, snaps *snapshot.Tree) (*StateDB, if err != nil && (sdb.snap == nil || snapVerified) { return nil, err } + _, sdb.noTrie = tr.(*trie.EmptyTrie) sdb.trie = tr return sdb, nil } @@ -204,6 +208,9 @@ func (s *StateDB) EnableWriteOnSharedStorage() { // state trie concurrently while the state is mutated so that when we reach the // commit phase, most of the needed data is already hot. func (s *StateDB) StartPrefetcher(namespace string) { + if s.noTrie { + return + } s.prefetcherLock.Lock() defer s.prefetcherLock.Unlock() if s.prefetcher != nil { @@ -218,6 +225,9 @@ func (s *StateDB) StartPrefetcher(namespace string) { // StopPrefetcher terminates a running prefetcher and reports any leftover stats // from the gathered metrics. func (s *StateDB) StopPrefetcher() { + if s.noTrie { + return + } s.prefetcherLock.Lock() defer s.prefetcherLock.Unlock() if s.prefetcher != nil { @@ -264,6 +274,10 @@ func (s *StateDB) setError(err error) { } } +func (s *StateDB) NoTrie() bool { + return s.noTrie +} + func (s *StateDB) Error() error { return s.dbErr } @@ -563,6 +577,9 @@ func (s *StateDB) Suicide(addr common.Address) bool { // updateStateObject writes the given object to the trie. func (s *StateDB) updateStateObject(obj *StateObject) { + if s.noTrie { + return + } // Track the amount of time wasted on updating the account from the trie if metrics.EnabledExpensive { defer func(start time.Time) { s.AccountUpdates += time.Since(start) }(time.Now()) @@ -576,6 +593,9 @@ func (s *StateDB) updateStateObject(obj *StateObject) { // deleteStateObject removes the given object from the state trie. func (s *StateDB) deleteStateObject(obj *StateObject) { + if s.noTrie { + return + } // Track the amount of time wasted on deleting the account from the trie if metrics.EnabledExpensive { defer func(start time.Time) { s.AccountUpdates += time.Since(start) }(time.Now()) @@ -634,6 +654,20 @@ func (s *StateDB) getDeletedStateObject(addr common.Address) *StateObject { } } } + + // ErrSnapshotStale may occur due to diff layers in the update, so we should try again in noTrie mode. + if s.NoTrie() && err != nil && errors.Is(err, snapshot.ErrSnapshotStale) { + // This error occurs when statedb.snaps.Cap changes the state of the merged difflayer + // to stale during the refresh of the difflayer, indicating that it is about to be discarded. + // Since the difflayer is refreshed in parallel, + // there is a small chance that the difflayer of the stale will be read while reading, + // resulting in an empty array being returned here. + // Therefore, noTrie mode must retry here, + // and add a time interval when retrying to avoid stacking too much and causing OOM. + time.Sleep(snapshotStaleRetryInterval) + return s.getDeletedStateObject(addr) + } + // If snapshot unavailable or reading from it failed, load from the database if s.snap == nil || err != nil { if s.trie == nil { @@ -883,9 +917,9 @@ func (s *StateDB) GetRefund() uint64 { return s.refund } -// GetRefund returns the current value of the refund counter. +// WaitPipeVerification waits until the snapshot been verified func (s *StateDB) WaitPipeVerification() error { - // We need wait for the parent trie to commit + // Need to wait for the parent trie to commit if s.snap != nil { if valid := s.snap.WaitAndGetVerifyRes(); !valid { return fmt.Errorf("verification on parent snap failed") @@ -947,6 +981,7 @@ func (s *StateDB) Finalise(deleteEmptyObjects bool) { // It is called in between transactions to get the root hash that // goes into transaction receipts. func (s *StateDB) IntermediateRoot(deleteEmptyObjects bool) common.Hash { + // light process is not allowed when there is no trie if s.lightProcessed { s.StopPrefetcher() return s.trie.Hash() @@ -972,7 +1007,7 @@ func (s *StateDB) CorrectAccountsRoot(blockRoot common.Hash) { if accounts, err := snapshot.Accounts(); err == nil && accounts != nil { for _, obj := range s.stateObjects { if !obj.deleted && !obj.rootCorrected && obj.data.Root == dummyRoot { - if account, exist := accounts[crypto.Keccak256Hash(obj.address[:])]; exist && len(account.Root) != 0 { + if account, exist := accounts[crypto.Keccak256Hash(obj.address[:])]; exist { obj.data.Root = common.BytesToHash(account.Root) obj.rootCorrected = true } @@ -985,7 +1020,7 @@ func (s *StateDB) CorrectAccountsRoot(blockRoot common.Hash) { func (s *StateDB) PopulateSnapAccountAndStorage() { for addr := range s.stateObjectsPending { if obj := s.stateObjects[addr]; !obj.deleted { - if s.snap != nil && !obj.deleted { + if s.snap != nil { root := obj.data.Root storageChanged := s.populateSnapStorage(obj) if storageChanged { @@ -1060,11 +1095,11 @@ func (s *StateDB) AccountsIntermediateRoot() { // update mechanism is not symmetric to the deletion, because whereas it is // enough to track account updates at commit time, deletions need tracking // at transaction boundary level to ensure we capture state clearing. - if s.snap != nil && !obj.deleted { - s.snapMux.Lock() + if s.snap != nil { + s.snapAccountMux.Lock() // It is possible to add unnecessary change, but it is fine. s.snapAccounts[obj.address] = snapshot.SlimAccountRLP(obj.data.Nonce, obj.data.Balance, obj.data.Root, obj.data.CodeHash) - s.snapMux.Unlock() + s.snapAccountMux.Unlock() } data, err := rlp.EncodeToBytes(obj) if err != nil { @@ -1114,18 +1149,18 @@ func (s *StateDB) StateIntermediateRoot() common.Hash { } usedAddrs := make([][]byte, 0, len(s.stateObjectsPending)) - for addr := range s.stateObjectsPending { - if obj := s.stateObjects[addr]; obj.deleted { - s.deleteStateObject(obj) - s.AccountDeleted += 1 - } else { - s.updateStateObject(obj) - s.AccountUpdated += 1 + if !s.noTrie { + for addr := range s.stateObjectsPending { + if obj := s.stateObjects[addr]; obj.deleted { + s.deleteStateObject(obj) + } else { + s.updateStateObject(obj) + } + usedAddrs = append(usedAddrs, common.CopyBytes(addr[:])) // Copy needed for closure + } + if prefetcher != nil { + prefetcher.used(s.originalRoot, usedAddrs) } - usedAddrs = append(usedAddrs, common.CopyBytes(addr[:])) // Copy needed for closure - } - if prefetcher != nil { - prefetcher.used(s.originalRoot, usedAddrs) } if len(s.stateObjectsPending) > 0 { @@ -1135,8 +1170,11 @@ func (s *StateDB) StateIntermediateRoot() common.Hash { if metrics.EnabledExpensive { defer func(start time.Time) { s.AccountHashes += time.Since(start) }(time.Now()) } - root := s.trie.Hash() - return root + if s.noTrie { + return s.expectedRoot + } else { + return s.trie.Hash() + } } // Prepare sets the current transaction hash and index which are @@ -1361,8 +1399,13 @@ func (s *StateDB) Commit(failPostCommitFunc func(), postCommitFuncs ...func() er // Write any contract code associated with the state object tasks <- func() { // Write any storage changes in the state object to its storage trie - _, err := obj.CommitTrie(s.db) - taskResults <- err + if !s.noTrie { + if _, err := obj.CommitTrie(s.db); err != nil { + taskResults <- err + return + } + } + taskResults <- nil } tasksNum++ } @@ -1379,24 +1422,27 @@ func (s *StateDB) Commit(failPostCommitFunc func(), postCommitFuncs ...func() er // The onleaf func is called _serially_, so we can reuse the same account // for unmarshalling every time. - var account types.StateAccount - root, _, err := s.trie.Commit(func(_ [][]byte, _ []byte, leaf []byte, parent common.Hash) error { - if err := rlp.DecodeBytes(leaf, &account); err != nil { + if !s.noTrie { + var account types.StateAccount + root, _, err := s.trie.Commit(func(_ [][]byte, _ []byte, leaf []byte, parent common.Hash) error { + if err := rlp.DecodeBytes(leaf, &account); err != nil { + return nil + } + if account.Root != emptyRoot { + s.db.TrieDB().Reference(account.Root, parent) + } return nil + }) + if err != nil { + return err } - if account.Root != emptyRoot { - s.db.TrieDB().Reference(account.Root, parent) + if root != emptyRoot { + s.db.CacheAccount(root, s.trie) } - return nil - }) - if err != nil { - return err - } - if root != emptyRoot { - s.db.CacheAccount(root, s.trie) } + for _, postFunc := range postCommitFuncs { - err = postFunc() + err := postFunc() if err != nil { return err } diff --git a/core/state_prefetcher.go b/core/state_prefetcher.go index 701c4c159d..d39f3e4e25 100644 --- a/core/state_prefetcher.go +++ b/core/state_prefetcher.go @@ -17,8 +17,6 @@ package core import ( - "sync/atomic" - "github.com/ethereum/go-ethereum/consensus" "github.com/ethereum/go-ethereum/core/state" "github.com/ethereum/go-ethereum/core/types" @@ -50,43 +48,45 @@ func NewStatePrefetcher(config *params.ChainConfig, bc *BlockChain, engine conse // Prefetch processes the state changes according to the Ethereum rules by running // the transaction messages using the statedb, but any changes are discarded. The // only goal is to pre-cache transaction signatures and snapshot clean state. -func (p *statePrefetcher) Prefetch(block *types.Block, statedb *state.StateDB, cfg vm.Config, interrupt *uint32) { +func (p *statePrefetcher) Prefetch(block *types.Block, statedb *state.StateDB, cfg *vm.Config, interruptCh <-chan struct{}) { var ( header = block.Header() signer = types.MakeSigner(p.config, header.Number) ) transactions := block.Transactions() - sortTransactions := make([][]*types.Transaction, prefetchThread) - for i := 0; i < prefetchThread; i++ { - sortTransactions[i] = make([]*types.Transaction, 0, len(transactions)/prefetchThread) - } - for idx := range transactions { - threadIdx := idx % prefetchThread - sortTransactions[threadIdx] = append(sortTransactions[threadIdx], transactions[idx]) - } + txChan := make(chan int, prefetchThread) // No need to execute the first batch, since the main processor will do it. for i := 0; i < prefetchThread; i++ { - go func(idx int) { + go func() { newStatedb := statedb.Copy() newStatedb.EnableWriteOnSharedStorage() gaspool := new(GasPool).AddGas(block.GasLimit()) blockContext := NewEVMBlockContext(header, p.bc, nil) - evm := vm.NewEVM(blockContext, vm.TxContext{}, statedb, p.config, cfg) + evm := vm.NewEVM(blockContext, vm.TxContext{}, statedb, p.config, *cfg) // Iterate over and process the individual transactions - for i, tx := range sortTransactions[idx] { - // If block precaching was interrupted, abort - if interrupt != nil && atomic.LoadUint32(interrupt) == 1 { + for { + select { + case txIndex := <-txChan: + tx := transactions[txIndex] + // Convert the transaction into an executable message and pre-cache its sender + msg, err := tx.AsMessageNoNonceCheck(signer) + if err != nil { + return // Also invalid block, bail out + } + newStatedb.Prepare(tx.Hash(), txIndex) + precacheTransaction(msg, p.config, gaspool, newStatedb, header, evm) + + case <-interruptCh: + // If block precaching was interrupted, abort return } - // Convert the transaction into an executable message and pre-cache its sender - msg, err := tx.AsMessage(signer, block.Header().BaseFee) - if err != nil { - return // Also invalid block, bail out - } - newStatedb.Prepare(tx.Hash(), i) - precacheTransaction(msg, p.config, gaspool, newStatedb, header, evm) } - }(i) + }() + } + + // it should be in a separate goroutine, to avoid blocking the critical path. + for i := 0; i < len(transactions); i++ { + txChan <- i } } @@ -127,23 +127,21 @@ func (p *statePrefetcher) PrefetchMining(txs *types.TransactionsByPriceAndNonce, go func(txset *types.TransactionsByPriceAndNonce) { count := 0 for { - tx := txset.Peek() - if tx == nil { - return - } select { case <-interruptCh: return default: - } - if count++; count%checkInterval == 0 { - if *txCurr == nil { + if count++; count%checkInterval == 0 { + txset.Forward(*txCurr) + } + tx := txset.Peek() + if tx == nil { return } - txset.Forward(*txCurr) + txCh <- tx + txset.Shift() + } - txCh <- tx - txset.Shift() } }(txs) } diff --git a/core/state_processor.go b/core/state_processor.go index 6190821bd8..97025fb9eb 100644 --- a/core/state_processor.go +++ b/core/state_processor.go @@ -118,7 +118,7 @@ func (p *LightStateProcessor) Process(block *types.Block, statedb *state.StateDB return statedb, receipts, logs, gasUsed, nil } log.Error("do light process err at block", "num", block.NumberU64(), "err", err) - p.bc.removeDiffLayers(diffLayer.DiffHash) + p.bc.removeDiffLayers(diffLayer.DiffHash.Load().(common.Hash)) // prepare new statedb statedb.StopPrefetcher() parent := p.bc.GetHeader(block.ParentHash(), block.NumberU64()-1) @@ -409,6 +409,7 @@ func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg // usually do have two tx, one for validator set contract, another for system reward contract. systemTxs := make([]*types.Transaction, 0, 2) + for i, tx := range block.Transactions() { if isPoSA { if isSystemTx, err := posa.IsSystemTransaction(tx, block.Header()); err != nil { @@ -426,13 +427,12 @@ func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg return statedb, nil, nil, 0, err } statedb.Prepare(tx.Hash(), i) - receipt, err := applyTransaction(msg, p.config, p.bc, nil, gp, statedb, blockNumber, blockHash, - tx, usedGas, vmenv, bloomProcessors) + + receipt, err := applyTransaction(msg, p.config, p.bc, nil, gp, statedb, blockNumber, blockHash, tx, usedGas, vmenv, bloomProcessors) if err != nil { bloomProcessors.Close() return statedb, nil, nil, 0, fmt.Errorf("could not apply tx %d [%v]: %w", i, tx.Hash().Hex(), err) } - commonTxs = append(commonTxs, tx) receipts = append(receipts, receipt) } diff --git a/core/state_processor_test.go b/core/state_processor_test.go index a914970c79..7ef8747806 100644 --- a/core/state_processor_test.go +++ b/core/state_processor_test.go @@ -18,6 +18,7 @@ package core import ( "crypto/ecdsa" + "errors" "math/big" "testing" @@ -241,13 +242,13 @@ func TestStateProcessorErrors(t *testing.T) { defer blockchain.Stop() for i, tt := range []struct { txs []*types.Transaction - want string + want error }{ { // ErrTxTypeNotSupported txs: []*types.Transaction{ mkDynamicTx(0, common.Address{}, params.TxGas-1000, big.NewInt(0), big.NewInt(0)), }, - want: "could not apply tx 0 [0x88626ac0d53cb65308f2416103c62bb1f18b805573d4f96a3640bbbfff13c14f]: transaction type not supported", + want: types.ErrTxTypeNotSupported, }, } { block := GenerateBadBlock(genesis, ethash.NewFaker(), tt.txs, gspec.Config) @@ -255,7 +256,7 @@ func TestStateProcessorErrors(t *testing.T) { if err == nil { t.Fatal("block imported without errors") } - if have, want := err.Error(), tt.want; have != want { + if have, want := err, tt.want; !errors.Is(have, want) { t.Errorf("test %d:\nhave \"%v\"\nwant \"%v\"\n", i, have, want) } } diff --git a/core/types.go b/core/types.go index c9061233e6..d0469dfcb6 100644 --- a/core/types.go +++ b/core/types.go @@ -32,6 +32,8 @@ type Validator interface { // ValidateState validates the given statedb and optionally the receipts and // gas used. ValidateState(block *types.Block, state *state.StateDB, receipts types.Receipts, usedGas uint64) error + // RemoteVerifyManager return remoteVerifyManager of validator. + RemoteVerifyManager() *remoteVerifyManager } // Prefetcher is an interface for pre-caching transaction signatures and state. @@ -39,7 +41,7 @@ type Prefetcher interface { // Prefetch processes the state changes according to the Ethereum rules by running // the transaction messages using the statedb, but any changes are discarded. The // only goal is to pre-cache transaction signatures and state trie nodes. - Prefetch(block *types.Block, statedb *state.StateDB, cfg vm.Config, interrupt *uint32) + Prefetch(block *types.Block, statedb *state.StateDB, cfg *vm.Config, interruptCh <-chan struct{}) // PrefetchMining used for pre-caching transaction signatures and state trie nodes. Only used for mining stage. PrefetchMining(txs *types.TransactionsByPriceAndNonce, header *types.Header, gasLimit uint64, statedb *state.StateDB, cfg vm.Config, interruptCh <-chan struct{}, txCurr **types.Transaction) } diff --git a/core/types/block.go b/core/types/block.go index 711587fa1d..59c6adf2a9 100644 --- a/core/types/block.go +++ b/core/types/block.go @@ -40,6 +40,32 @@ var ( EmptyUncleHash = rlpHash([]*Header(nil)) ) +type VerifyStatus struct { + Code uint16 + Msg string +} + +var ( + // StatusVerified means the processing of request going as expected and found the root correctly. + StatusVerified = VerifyStatus{Code: 0x100} + StatusFullVerified = VerifyStatus{Code: 0x101, Msg: "state root full verified"} + StatusPartiallyVerified = VerifyStatus{Code: 0x102, Msg: "state root partially verified, because of difflayer not found"} + + // StatusFailed means the request has something wrong. + StatusFailed = VerifyStatus{Code: 0x200} + StatusDiffHashMismatch = VerifyStatus{Code: 0x201, Msg: "verify failed because of blockhash mismatch with diffhash"} + StatusImpossibleFork = VerifyStatus{Code: 0x202, Msg: "verify failed because of impossible fork detected"} + + // StatusUncertain means verify node can't give a certain result of the request. + StatusUncertain = VerifyStatus{Code: 0x300} + StatusBlockTooNew = VerifyStatus{Code: 0x301, Msg: "can’t verify because of block number larger than current height more than 11"} + StatusBlockNewer = VerifyStatus{Code: 0x302, Msg: "can’t verify because of block number larger than current height"} + StatusPossibleFork = VerifyStatus{Code: 0x303, Msg: "can’t verify because of possible fork detected"} + + // StatusUnexpectedError is unexpected internal error. + StatusUnexpectedError = VerifyStatus{Code: 0x400, Msg: "can’t verify because of unexpected internal error"} +) + // A BlockNonce is a 64-bit hash which proves (combined with the // mix-hash) that a sufficient amount of computation has been carried // out on a block. @@ -423,10 +449,10 @@ type DiffLayer struct { Accounts []DiffAccount Storages []DiffStorage - DiffHash common.Hash + DiffHash atomic.Value } -type extDiffLayer struct { +type ExtDiffLayer struct { BlockHash common.Hash Number uint64 Receipts []*ReceiptForStorage // Receipts are duplicated stored to simplify the logic @@ -438,7 +464,7 @@ type extDiffLayer struct { // DecodeRLP decodes the Ethereum func (d *DiffLayer) DecodeRLP(s *rlp.Stream) error { - var ed extDiffLayer + var ed ExtDiffLayer if err := s.Decode(&ed); err != nil { return err } @@ -458,7 +484,7 @@ func (d *DiffLayer) EncodeRLP(w io.Writer) error { for i, receipt := range d.Receipts { storageReceipts[i] = (*ReceiptForStorage)(receipt) } - return rlp.Encode(w, extDiffLayer{ + return rlp.Encode(w, ExtDiffLayer{ BlockHash: d.BlockHash, Number: d.Number, Receipts: storageReceipts, @@ -497,6 +523,13 @@ type DiffStorage struct { Vals [][]byte } +func (storage *DiffStorage) Len() int { return len(storage.Keys) } +func (storage *DiffStorage) Swap(i, j int) { + storage.Keys[i], storage.Keys[j] = storage.Keys[j], storage.Keys[i] + storage.Vals[i], storage.Vals[j] = storage.Vals[j], storage.Vals[i] +} +func (storage *DiffStorage) Less(i, j int) bool { return storage.Keys[i] < storage.Keys[j] } + type DiffAccountsInTx struct { TxHash common.Hash Accounts map[common.Address]*big.Int diff --git a/core/types/transaction.go b/core/types/transaction.go index 612e4df847..1e6c45cd7f 100644 --- a/core/types/transaction.go +++ b/core/types/transaction.go @@ -596,7 +596,9 @@ func (t *TransactionsByPriceAndNonce) CurrentSize() int { //Forward moves current transaction to be the one which is one index after tx func (t *TransactionsByPriceAndNonce) Forward(tx *Transaction) { if tx == nil { - t.heads = t.heads[0:0] + if len(t.heads) > 0 { + t.heads = t.heads[0:0] + } return } //check whether target tx exists in t.heads diff --git a/core/types/transaction_test.go b/core/types/transaction_test.go index 86c0dffc2c..e88dd3e852 100644 --- a/core/types/transaction_test.go +++ b/core/types/transaction_test.go @@ -438,7 +438,7 @@ func TestTransactionForward(t *testing.T) { } tmp := txset.Copy() - for j := 0; j < 10; j++ { + for j := 0; j < 11; j++ { txset = tmp.Copy() txsetCpy = tmp.Copy() i := 0 @@ -446,6 +446,9 @@ func TestTransactionForward(t *testing.T) { txset.Shift() } tx := txset.Peek() + if tx == nil { + continue + } txsetCpy.Forward(tx) txCpy := txsetCpy.Peek() if txCpy == nil { diff --git a/core/vm/interpreter.go b/core/vm/interpreter.go index f4fed969de..030a063948 100644 --- a/core/vm/interpreter.go +++ b/core/vm/interpreter.go @@ -98,15 +98,19 @@ func NewEVMInterpreter(evm *EVM, cfg Config) *EVMInterpreter { default: cfg.JumpTable = &frontierInstructionSet } - for i, eip := range cfg.ExtraEips { + var extraEips []int + for _, eip := range cfg.ExtraEips { copy := *cfg.JumpTable if err := EnableEIP(eip, ©); err != nil { // Disable it, so caller can check if it's activated or not - cfg.ExtraEips = append(cfg.ExtraEips[:i], cfg.ExtraEips[i+1:]...) log.Error("EIP activation failed", "eip", eip, "error", err) + } else { + extraEips = append(extraEips, eip) } cfg.JumpTable = © } + cfg.ExtraEips = extraEips + } evmInterpreter := EVMInterpreterPool.Get().(*EVMInterpreter) evmInterpreter.evm = evm diff --git a/docs/parlia/README-BEP-127.md b/docs/parlia/README-BEP-127.md new file mode 100644 index 0000000000..f79b07141f --- /dev/null +++ b/docs/parlia/README-BEP-127.md @@ -0,0 +1,47 @@ +## BEP-127: Temporary Maintenance Mode for Validators + +Temporary Maintenance is supposed to last one or a few hours. The validator seat will be temporarily dropped from the block producing rotation during the maintenance. Since long-time offline maintenance is not encouraged, the validator will still be slashed if the maintenance lasts too long. To lower the impact from poorly-operating validators who forget to claim its maintenance, they will be forced to enter Temporary Maintenance mode too. + +- **enterMaintenance**: Validator can claim itself to enter scheduled maintenance by sending a transaction signed by the consensus key. +- **exitMaintenance**: The validator can claim itself to exit maintenance by sending another transaction. + +More details in [BEP-127](https://github.com/bnb-chain/BEPs/blob/master/BEP127.md). + + +## How to enter/exit maintenance + +### Running `geth` +make sure you have unlocked the consensus address of your validator + +### Running `built-in interactive` +```shell +$ geth attach geth.ipc +``` + +This command will: +* Start up `geth`'s built-in interactive [JavaScript console](https://geth.ethereum.org/docs/interface/javascript-console), + (via the trailing `console` subcommand) through which you can interact using [`web3` methods](https://web3js.readthedocs.io/en/) + (note: the `web3` version bundled within `geth` is very old, and not up to date with official docs), + as well as `geth`'s own [management APIs](https://geth.ethereum.org/docs/rpc/server). + + +### enter maintenance +``` +web3.eth.sendTransaction({ + from: "consensus address of your validator", + to: "0x0000000000000000000000000000000000001000", + data: "0x9369d7de" +}) +``` + +### exit maintenance +``` +web3.eth.sendTransaction({ + from: "consensus address of your validator", + to: "0x0000000000000000000000000000000000001000", + data: "0x04c4fec6" +}) +``` + + + diff --git a/eth/backend.go b/eth/backend.go index 6b286d0c32..e9c077e6a7 100644 --- a/eth/backend.go +++ b/eth/backend.go @@ -46,6 +46,7 @@ import ( "github.com/ethereum/go-ethereum/eth/protocols/diff" "github.com/ethereum/go-ethereum/eth/protocols/eth" "github.com/ethereum/go-ethereum/eth/protocols/snap" + "github.com/ethereum/go-ethereum/eth/protocols/trust" "github.com/ethereum/go-ethereum/ethdb" "github.com/ethereum/go-ethereum/event" "github.com/ethereum/go-ethereum/internal/ethapi" @@ -70,12 +71,13 @@ type Ethereum struct { config *ethconfig.Config // Handlers - txPool *core.TxPool - blockchain *core.BlockChain - handler *handler - ethDialCandidates enode.Iterator - snapDialCandidates enode.Iterator - merger *consensus.Merger + txPool *core.TxPool + blockchain *core.BlockChain + handler *handler + ethDialCandidates enode.Iterator + snapDialCandidates enode.Iterator + trustDialCandidates enode.Iterator + merger *consensus.Merger // DB interfaces chainDb ethdb.Database // Block chain database @@ -114,6 +116,9 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) { if !config.SyncMode.IsValid() { return nil, fmt.Errorf("invalid sync mode %d", config.SyncMode) } + if !config.TriesVerifyMode.IsValid() { + return nil, fmt.Errorf("invalid tries verify mode %d", config.TriesVerifyMode) + } if config.Miner.GasPrice == nil || config.Miner.GasPrice.Cmp(common.Big0) <= 0 { log.Warn("Sanitizing invalid miner gas price", "provided", config.Miner.GasPrice, "updated", ethconfig.Defaults.Miner.GasPrice) config.Miner.GasPrice = new(big.Int).Set(ethconfig.Defaults.Miner.GasPrice) @@ -135,7 +140,7 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) { // Assemble the Ethereum object chainDb, err := stack.OpenAndMergeDatabase("chaindata", config.DatabaseCache, config.DatabaseHandles, - config.DatabaseFreezer, config.DatabaseDiff, "eth/db/chaindata/", false, config.PersistDiff) + config.DatabaseFreezer, config.DatabaseDiff, "eth/db/chaindata/", false, config.PersistDiff, config.PruneAncientData) if err != nil { return nil, err } @@ -200,14 +205,14 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) { TrieDirtyLimit: config.TrieDirtyCache, TrieDirtyDisabled: config.NoPruning, TrieTimeLimit: config.TrieTimeout, + NoTries: config.TriesVerifyMode != core.LocalVerify, SnapshotLimit: config.SnapshotCache, TriesInMemory: config.TriesInMemory, Preimages: config.Preimages, } ) bcOps := make([]core.BlockChainOption, 0) - // TODO diffsync performance is not as expected, disable it when pipecommit is enabled for now - if config.DiffSync && !config.PipeCommit { + if config.DiffSync && !config.PipeCommit && config.TriesVerifyMode == core.LocalVerify { bcOps = append(bcOps, core.EnableLightProcessor) } if config.PipeCommit { @@ -216,6 +221,9 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) { if config.PersistDiff { bcOps = append(bcOps, core.EnablePersistDiff(config.DiffBlock)) } + + peers := newPeerSet() + bcOps = append(bcOps, core.EnableBlockValidator(chainConfig, eth.engine, config.TriesVerifyMode, peers)) eth.blockchain, err = core.NewBlockChain(chainDb, cacheConfig, chainConfig, eth.engine, vmConfig, eth.shouldPreserve, &config.TxLookupLimit, bcOps...) if err != nil { return nil, err @@ -254,6 +262,7 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) { DirectBroadcast: config.DirectBroadcast, DiffSync: config.DiffSync, DisablePeerTxBroadcast: config.DisablePeerTxBroadcast, + PeerSet: peers, }); err != nil { return nil, err } @@ -277,6 +286,10 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) { if err != nil { return nil, err } + eth.trustDialCandidates, err = dnsclient.NewIterator(eth.config.TrustDiscoveryURLs...) + if err != nil { + return nil, err + } // Start the RPC service eth.netRPCService = ethapi.NewPublicNetAPI(eth.p2pServer, config.NetworkId) @@ -565,7 +578,12 @@ func (s *Ethereum) Protocols() []p2p.Protocol { protos = append(protos, snap.MakeProtocols((*snapHandler)(s.handler), s.snapDialCandidates)...) } // diff protocol can still open without snap protocol - protos = append(protos, diff.MakeProtocols((*diffHandler)(s.handler), s.snapDialCandidates)...) + if !s.config.DisableDiffProtocol { + protos = append(protos, diff.MakeProtocols((*diffHandler)(s.handler), s.snapDialCandidates)...) + } + if s.config.EnableTrustProtocol { + protos = append(protos, trust.MakeProtocols((*trustHandler)(s.handler), s.snapDialCandidates)...) + } return protos } @@ -599,6 +617,7 @@ func (s *Ethereum) Stop() error { // Stop all the peer-related stuff first. s.ethDialCandidates.Close() s.snapDialCandidates.Close() + s.trustDialCandidates.Close() s.handler.Stop() // Then stop everything else. diff --git a/eth/downloader/downloader.go b/eth/downloader/downloader.go index e0ed4fc320..207ed39915 100644 --- a/eth/downloader/downloader.go +++ b/eth/downloader/downloader.go @@ -27,6 +27,7 @@ import ( "github.com/ethereum/go-ethereum" "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core" "github.com/ethereum/go-ethereum/core/rawdb" "github.com/ethereum/go-ethereum/core/state/snapshot" "github.com/ethereum/go-ethereum/core/types" @@ -1451,6 +1452,9 @@ func (d *Downloader) importBlockResults(results []*fetchResult) error { // of the blocks delivered from the downloader, and the indexing will be off. log.Debug("Downloaded item processing failed on sidechain import", "index", index, "err", err) } + if errors.Is(err, core.ErrAncestorHasNotBeenVerified) { + return err + } return fmt.Errorf("%w: %v", errInvalidChain, err) } return nil diff --git a/eth/downloader/downloader_test.go b/eth/downloader/downloader_test.go index cc4970848a..a2e0e2e34d 100644 --- a/eth/downloader/downloader_test.go +++ b/eth/downloader/downloader_test.go @@ -60,7 +60,7 @@ func newTester() *downloadTester { if err != nil { panic(err) } - db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), freezer, "", false, false, false) + db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), freezer, "", false, false, false, false) if err != nil { panic(err) } diff --git a/eth/ethconfig/config.go b/eth/ethconfig/config.go index 295b1c49c1..03104c6109 100644 --- a/eth/ethconfig/config.go +++ b/eth/ethconfig/config.go @@ -82,6 +82,7 @@ var Defaults = Config{ TrieDirtyCache: 256, TrieTimeout: 60 * time.Minute, TriesInMemory: 128, + TriesVerifyMode: core.LocalVerify, SnapshotCache: 102, DiffBlock: uint64(86400), Miner: miner.Config{ @@ -133,12 +134,15 @@ type Config struct { // This can be set to list of enrtree:// URLs which will be queried for // for nodes to connect to. - EthDiscoveryURLs []string - SnapDiscoveryURLs []string + EthDiscoveryURLs []string + SnapDiscoveryURLs []string + TrustDiscoveryURLs []string NoPruning bool // Whether to disable pruning and flush everything to disk DirectBroadcast bool DisableSnapProtocol bool //Whether disable snap protocol + DisableDiffProtocol bool //Whether disable diff protocol + EnableTrustProtocol bool //Whether enable trust protocol DiffSync bool // Whether support diff sync PipeCommit bool RangeLimit bool @@ -170,6 +174,7 @@ type Config struct { DatabaseDiff string PersistDiff bool DiffBlock uint64 + PruneAncientData bool TrieCleanCache int TrieCleanCacheJournal string `toml:",omitempty"` // Disk journal directory for trie cache to survive node restarts @@ -178,6 +183,7 @@ type Config struct { TrieTimeout time.Duration SnapshotCache int TriesInMemory uint64 + TriesVerifyMode core.VerifyMode Preimages bool // Mining options diff --git a/eth/ethconfig/gen_config.go b/eth/ethconfig/gen_config.go index e26ffa5222..c0d44ce141 100644 --- a/eth/ethconfig/gen_config.go +++ b/eth/ethconfig/gen_config.go @@ -18,53 +18,64 @@ import ( // MarshalTOML marshals as TOML. func (c Config) MarshalTOML() (interface{}, error) { type Config struct { - Genesis *core.Genesis `toml:",omitempty"` - NetworkId uint64 - SyncMode downloader.SyncMode - DisablePeerTxBroadcast bool - EthDiscoveryURLs []string - SnapDiscoveryURLs []string - NoPruning bool - NoPrefetch bool - TxLookupLimit uint64 `toml:",omitempty"` - Whitelist map[uint64]common.Hash `toml:"-"` - LightServ int `toml:",omitempty"` - LightIngress int `toml:",omitempty"` - LightEgress int `toml:",omitempty"` - LightPeers int `toml:",omitempty"` - LightNoPrune bool `toml:",omitempty"` - LightNoSyncServe bool `toml:",omitempty"` - SyncFromCheckpoint bool `toml:",omitempty"` - UltraLightServers []string `toml:",omitempty"` - UltraLightFraction int `toml:",omitempty"` - UltraLightOnlyAnnounce bool `toml:",omitempty"` - SkipBcVersionCheck bool `toml:"-"` - DatabaseHandles int `toml:"-"` - DatabaseCache int - DatabaseFreezer string - DatabaseDiff string - TrieCleanCache int - TrieCleanCacheJournal string `toml:",omitempty"` - TrieCleanCacheRejournal time.Duration `toml:",omitempty"` - TrieDirtyCache int - TrieTimeout time.Duration - TriesInMemory uint64 `toml:",omitempty"` - SnapshotCache int - Preimages bool - PersistDiff bool - DiffBlock uint64 `toml:",omitempty"` - Miner miner.Config - Ethash ethash.Config - TxPool core.TxPoolConfig - GPO gasprice.Config - EnablePreimageRecording bool - DocRoot string `toml:"-"` - RPCGasCap uint64 - RPCEVMTimeout time.Duration - RPCTxFeeCap float64 - Checkpoint *params.TrustedCheckpoint `toml:",omitempty"` - CheckpointOracle *params.CheckpointOracleConfig `toml:",omitempty"` - OverrideBerlin *big.Int `toml:",omitempty"` + Genesis *core.Genesis `toml:",omitempty"` + NetworkId uint64 + SyncMode downloader.SyncMode + DisablePeerTxBroadcast bool + EthDiscoveryURLs []string + SnapDiscoveryURLs []string + TrustDiscoveryURLs []string + NoPruning bool + NoPrefetch bool + DirectBroadcast bool + DisableSnapProtocol bool + DisableDiffProtocol bool + EnableTrustProtocol bool + DiffSync bool + RangeLimit bool + TxLookupLimit uint64 `toml:",omitempty"` + Whitelist map[uint64]common.Hash `toml:"-"` + LightServ int `toml:",omitempty"` + LightIngress int `toml:",omitempty"` + LightEgress int `toml:",omitempty"` + LightPeers int `toml:",omitempty"` + LightNoPrune bool `toml:",omitempty"` + LightNoSyncServe bool `toml:",omitempty"` + SyncFromCheckpoint bool `toml:",omitempty"` + UltraLightServers []string `toml:",omitempty"` + UltraLightFraction int `toml:",omitempty"` + UltraLightOnlyAnnounce bool `toml:",omitempty"` + SkipBcVersionCheck bool `toml:"-"` + DatabaseHandles int `toml:"-"` + DatabaseCache int + DatabaseFreezer string + DatabaseDiff string + TrieCleanCache int + TrieCleanCacheJournal string `toml:",omitempty"` + TrieCleanCacheRejournal time.Duration `toml:",omitempty"` + TrieDirtyCache int + TrieTimeout time.Duration + SnapshotCache int + TriesInMemory uint64 + TriesVerifyMode core.VerifyMode + Preimages bool + PersistDiff bool + DiffBlock uint64 `toml:",omitempty"` + PruneAncientData bool + Miner miner.Config + Ethash ethash.Config `toml:",omitempty"` + TxPool core.TxPoolConfig + GPO gasprice.Config + EnablePreimageRecording bool + DocRoot string `toml:"-"` + EWASMInterpreter string + EVMInterpreter string + RPCGasCap uint64 + RPCEVMTimeout time.Duration + RPCTxFeeCap float64 + Checkpoint *params.TrustedCheckpoint `toml:",omitempty"` + CheckpointOracle *params.CheckpointOracleConfig `toml:",omitempty"` + OverrideBerlin *big.Int `toml:",omitempty"` OverrideArrowGlacier *big.Int `toml:",omitempty"` OverrideTerminalTotalDifficulty *big.Int `toml:",omitempty"` } @@ -75,7 +86,14 @@ func (c Config) MarshalTOML() (interface{}, error) { enc.DisablePeerTxBroadcast = c.DisablePeerTxBroadcast enc.EthDiscoveryURLs = c.EthDiscoveryURLs enc.SnapDiscoveryURLs = c.SnapDiscoveryURLs + enc.TrustDiscoveryURLs = c.TrustDiscoveryURLs enc.NoPruning = c.NoPruning + enc.DirectBroadcast = c.DirectBroadcast + enc.DisableSnapProtocol = c.DisableSnapProtocol + enc.DisableDiffProtocol = c.DisableDiffProtocol + enc.EnableTrustProtocol = c.EnableTrustProtocol + enc.DiffSync = c.DiffSync + enc.RangeLimit = c.RangeLimit enc.TxLookupLimit = c.TxLookupLimit enc.Whitelist = c.Whitelist enc.LightServ = c.LightServ @@ -93,16 +111,20 @@ func (c Config) MarshalTOML() (interface{}, error) { enc.DatabaseCache = c.DatabaseCache enc.DatabaseFreezer = c.DatabaseFreezer enc.DatabaseDiff = c.DatabaseDiff + enc.PersistDiff = c.PersistDiff + enc.DiffBlock = c.DiffBlock enc.TrieCleanCache = c.TrieCleanCache enc.TrieCleanCacheJournal = c.TrieCleanCacheJournal enc.TrieCleanCacheRejournal = c.TrieCleanCacheRejournal enc.TrieDirtyCache = c.TrieDirtyCache enc.TrieTimeout = c.TrieTimeout - enc.TriesInMemory = c.TriesInMemory enc.SnapshotCache = c.SnapshotCache + enc.TriesInMemory = c.TriesInMemory + enc.TriesVerifyMode = c.TriesVerifyMode enc.Preimages = c.Preimages enc.PersistDiff = c.PersistDiff enc.DiffBlock = c.DiffBlock + enc.PruneAncientData = c.PruneAncientData enc.Miner = c.Miner enc.Ethash = c.Ethash enc.TxPool = c.TxPool @@ -123,53 +145,64 @@ func (c Config) MarshalTOML() (interface{}, error) { // UnmarshalTOML unmarshals from TOML. func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error { type Config struct { - Genesis *core.Genesis `toml:",omitempty"` - NetworkId *uint64 - SyncMode *downloader.SyncMode - DisablePeerTxBroadcast *bool - EthDiscoveryURLs []string - SnapDiscoveryURLs []string - NoPruning *bool - NoPrefetch *bool - TxLookupLimit *uint64 `toml:",omitempty"` - Whitelist map[uint64]common.Hash `toml:"-"` - LightServ *int `toml:",omitempty"` - LightIngress *int `toml:",omitempty"` - LightEgress *int `toml:",omitempty"` - LightPeers *int `toml:",omitempty"` - LightNoPrune *bool `toml:",omitempty"` - LightNoSyncServe *bool `toml:",omitempty"` - SyncFromCheckpoint *bool `toml:",omitempty"` - UltraLightServers []string `toml:",omitempty"` - UltraLightFraction *int `toml:",omitempty"` - UltraLightOnlyAnnounce *bool `toml:",omitempty"` - SkipBcVersionCheck *bool `toml:"-"` - DatabaseHandles *int `toml:"-"` - DatabaseCache *int - DatabaseFreezer *string - DatabaseDiff *string - PersistDiff *bool - DiffBlock *uint64 `toml:",omitempty"` - TrieCleanCache *int - TrieCleanCacheJournal *string `toml:",omitempty"` - TrieCleanCacheRejournal *time.Duration `toml:",omitempty"` - TrieDirtyCache *int - TrieTimeout *time.Duration - TriesInMemory *uint64 `toml:",omitempty"` - SnapshotCache *int - Preimages *bool - Miner *miner.Config - Ethash *ethash.Config - TxPool *core.TxPoolConfig - GPO *gasprice.Config - EnablePreimageRecording *bool - DocRoot *string `toml:"-"` - RPCGasCap *uint64 + Genesis *core.Genesis `toml:",omitempty"` + NetworkId *uint64 + SyncMode *downloader.SyncMode + DisablePeerTxBroadcast *bool + EthDiscoveryURLs []string + SnapDiscoveryURLs []string + TrustDiscoveryURLs []string + NoPruning *bool + NoPrefetch *bool + DirectBroadcast *bool + DisableSnapProtocol *bool + DisableDiffProtocol *bool + EnableTrustProtocol *bool + DiffSync *bool + RangeLimit *bool + TxLookupLimit *uint64 `toml:",omitempty"` + Whitelist map[uint64]common.Hash `toml:"-"` + LightServ *int `toml:",omitempty"` + LightIngress *int `toml:",omitempty"` + LightEgress *int `toml:",omitempty"` + LightPeers *int `toml:",omitempty"` + LightNoPrune *bool `toml:",omitempty"` + LightNoSyncServe *bool `toml:",omitempty"` + SyncFromCheckpoint *bool `toml:",omitempty"` + UltraLightServers []string `toml:",omitempty"` + UltraLightFraction *int `toml:",omitempty"` + UltraLightOnlyAnnounce *bool `toml:",omitempty"` + SkipBcVersionCheck *bool `toml:"-"` + DatabaseHandles *int `toml:"-"` + DatabaseCache *int + DatabaseFreezer *string + DatabaseDiff *string + PersistDiff *bool + DiffBlock *uint64 `toml:",omitempty"` + PruneAncientData *bool + TrieCleanCache *int + TrieCleanCacheJournal *string `toml:",omitempty"` + TrieCleanCacheRejournal *time.Duration `toml:",omitempty"` + TrieDirtyCache *int + TrieTimeout *time.Duration + SnapshotCache *int + TriesInMemory *uint64 + TriesVerifyMode *core.VerifyMode + Preimages *bool + Miner *miner.Config + Ethash *ethash.Config `toml:",omitempty"` + TxPool *core.TxPoolConfig + GPO *gasprice.Config + EnablePreimageRecording *bool + DocRoot *string `toml:"-"` + EWASMInterpreter *string + EVMInterpreter *string + RPCGasCap *uint64 RPCEVMTimeout *time.Duration - RPCTxFeeCap *float64 - Checkpoint *params.TrustedCheckpoint `toml:",omitempty"` - CheckpointOracle *params.CheckpointOracleConfig `toml:",omitempty"` - OverrideBerlin *big.Int `toml:",omitempty"` + RPCTxFeeCap *float64 + Checkpoint *params.TrustedCheckpoint `toml:",omitempty"` + CheckpointOracle *params.CheckpointOracleConfig `toml:",omitempty"` + OverrideBerlin *big.Int `toml:",omitempty"` OverrideArrowGlacier *big.Int `toml:",omitempty"` OverrideTerminalTotalDifficulty *big.Int `toml:",omitempty"` } @@ -195,9 +228,30 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error { if dec.SnapDiscoveryURLs != nil { c.SnapDiscoveryURLs = dec.SnapDiscoveryURLs } + if dec.TrustDiscoveryURLs != nil { + c.TrustDiscoveryURLs = dec.TrustDiscoveryURLs + } if dec.NoPruning != nil { c.NoPruning = *dec.NoPruning } + if dec.DirectBroadcast != nil { + c.DirectBroadcast = *dec.DirectBroadcast + } + if dec.DisableSnapProtocol != nil { + c.DisableSnapProtocol = *dec.DisableSnapProtocol + } + if dec.DisableDiffProtocol != nil { + c.DisableDiffProtocol = *dec.DisableDiffProtocol + } + if dec.EnableTrustProtocol != nil { + c.EnableTrustProtocol = *dec.EnableTrustProtocol + } + if dec.DiffSync != nil { + c.DiffSync = *dec.DiffSync + } + if dec.RangeLimit != nil { + c.RangeLimit = *dec.RangeLimit + } if dec.TxLookupLimit != nil { c.TxLookupLimit = *dec.TxLookupLimit } @@ -255,6 +309,9 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error { if dec.DiffBlock != nil { c.DiffBlock = *dec.DiffBlock } + if dec.PruneAncientData != nil { + c.PruneAncientData = *dec.PruneAncientData + } if dec.TrieCleanCache != nil { c.TrieCleanCache = *dec.TrieCleanCache } @@ -270,11 +327,14 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error { if dec.TrieTimeout != nil { c.TrieTimeout = *dec.TrieTimeout } + if dec.SnapshotCache != nil { + c.SnapshotCache = *dec.SnapshotCache + } if dec.TriesInMemory != nil { c.TriesInMemory = *dec.TriesInMemory } - if dec.SnapshotCache != nil { - c.SnapshotCache = *dec.SnapshotCache + if dec.TriesVerifyMode != nil { + c.TriesVerifyMode = *dec.TriesVerifyMode } if dec.Preimages != nil { c.Preimages = *dec.Preimages diff --git a/eth/handler.go b/eth/handler.go index 41d523e574..62befdbd09 100644 --- a/eth/handler.go +++ b/eth/handler.go @@ -29,12 +29,14 @@ import ( "github.com/ethereum/go-ethereum/consensus/beacon" "github.com/ethereum/go-ethereum/core" "github.com/ethereum/go-ethereum/core/forkid" + "github.com/ethereum/go-ethereum/core/rawdb" "github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/eth/downloader" "github.com/ethereum/go-ethereum/eth/fetcher" "github.com/ethereum/go-ethereum/eth/protocols/diff" "github.com/ethereum/go-ethereum/eth/protocols/eth" "github.com/ethereum/go-ethereum/eth/protocols/snap" + "github.com/ethereum/go-ethereum/eth/protocols/trust" "github.com/ethereum/go-ethereum/ethdb" "github.com/ethereum/go-ethereum/event" "github.com/ethereum/go-ethereum/log" @@ -96,6 +98,7 @@ type handlerConfig struct { Whitelist map[uint64]common.Hash // Hard coded whitelist for sync challenged DirectBroadcast bool DisablePeerTxBroadcast bool + PeerSet *peerSet } type handler struct { @@ -145,6 +148,9 @@ func newHandler(config *handlerConfig) (*handler, error) { if config.EventMux == nil { config.EventMux = new(event.TypeMux) // Nicety initialization for tests } + if config.PeerSet == nil { + config.PeerSet = newPeerSet() // Nicety initialization for tests + } h := &handler{ networkID: config.Network, forkFilter: forkid.NewFilter(config.Chain), @@ -153,7 +159,7 @@ func newHandler(config *handlerConfig) (*handler, error) { database: config.Database, txpool: config.TxPool, chain: config.Chain, - peers: newPeerSet(), + peers: config.PeerSet, merger: config.Merger, whitelist: config.Whitelist, directBroadcast: config.DirectBroadcast, @@ -171,6 +177,9 @@ func newHandler(config *handlerConfig) (*handler, error) { // In these cases however it's safe to reenable snap sync. fullBlock, fastBlock := h.chain.CurrentBlock(), h.chain.CurrentFastBlock() if fullBlock.NumberU64() == 0 && fastBlock.NumberU64() > 0 { + if rawdb.ReadAncientType(h.database) == rawdb.PruneFreezerType { + log.Crit("Fast Sync not finish, can't enable pruneancient mode") + } h.snapSync = uint32(1) log.Warn("Switch sync mode from full sync to snap sync") } @@ -311,6 +320,11 @@ func (h *handler) runEthPeer(peer *eth.Peer, handler eth.Handler) error { peer.Log().Error("Diff extension barrier failed", "err", err) return err } + trust, err := h.peers.waitTrustExtension(peer) + if err != nil { + peer.Log().Error("Trust extension barrier failed", "err", err) + return err + } // TODO(karalabe): Not sure why this is needed if !h.chainSync.handlePeerEvent(peer) { return p2p.DiscQuitting @@ -351,7 +365,7 @@ func (h *handler) runEthPeer(peer *eth.Peer, handler eth.Handler) error { peer.Log().Debug("Ethereum peer connected", "name", peer.Name()) // Register the peer locally - if err := h.peers.registerPeer(peer, snap, diff); err != nil { + if err := h.peers.registerPeer(peer, snap, diff, trust); err != nil { peer.Log().Error("Ethereum peer registration failed", "err", err) return err } @@ -500,6 +514,21 @@ func (h *handler) runDiffExtension(peer *diff.Peer, handler diff.Handler) error return handler(peer) } +// runTrustExtension registers a `trust` peer into the joint eth/trust peerset and +// starts handling inbound messages. As `trust` is only a satellite protocol to +// `eth`, all subsystem registrations and lifecycle management will be done by +// the main `eth` handler to prevent strange races. +func (h *handler) runTrustExtension(peer *trust.Peer, handler trust.Handler) error { + h.peerWG.Add(1) + defer h.peerWG.Done() + + if err := h.peers.registerTrustExtension(peer); err != nil { + peer.Log().Error("Trust extension registration failed", "err", err) + return err + } + return handler(peer) +} + // removePeer requests disconnection of a peer. func (h *handler) removePeer(id string) { peer := h.peers.peer(id) diff --git a/eth/handler_trust.go b/eth/handler_trust.go new file mode 100644 index 0000000000..0b116b9255 --- /dev/null +++ b/eth/handler_trust.go @@ -0,0 +1,52 @@ +package eth + +import ( + "fmt" + + "github.com/ethereum/go-ethereum/core" + "github.com/ethereum/go-ethereum/eth/protocols/trust" + "github.com/ethereum/go-ethereum/p2p/enode" +) + +// trustHandler implements the trust.Backend interface to handle the various network +// packets that are sent as replies or broadcasts. +type trustHandler handler + +func (h *trustHandler) Chain() *core.BlockChain { return h.chain } + +// RunPeer is invoked when a peer joins on the `snap` protocol. +func (h *trustHandler) RunPeer(peer *trust.Peer, hand trust.Handler) error { + return (*handler)(h).runTrustExtension(peer, hand) +} + +// PeerInfo retrieves all known `trust` information about a peer. +func (h *trustHandler) PeerInfo(id enode.ID) interface{} { + if p := h.peers.peer(id.String()); p != nil { + if p.trustExt != nil { + return p.trustExt.info() + } + } + return nil +} + +// Handle is invoked from a peer's message handler when it receives a new remote +// message that the handler couldn't consume and serve itself. +func (h *trustHandler) Handle(peer *trust.Peer, packet trust.Packet) error { + switch packet := packet.(type) { + case *trust.RootResponsePacket: + verifyResult := &core.VerifyResult{ + Status: packet.Status, + BlockNumber: packet.BlockNumber, + BlockHash: packet.BlockHash, + Root: packet.Root, + } + if vm := h.Chain().Validator().RemoteVerifyManager(); vm != nil { + vm.HandleRootResponse(verifyResult, peer.ID()) + return nil + } + return fmt.Errorf("verify manager is nil which is unexpected") + + default: + return fmt.Errorf("unexpected trust packet type: %T", packet) + } +} diff --git a/eth/peer.go b/eth/peer.go index 83acf98e9f..802d1e5aac 100644 --- a/eth/peer.go +++ b/eth/peer.go @@ -20,6 +20,8 @@ import ( "math/big" "time" + "github.com/ethereum/go-ethereum/eth/protocols/trust" + "github.com/ethereum/go-ethereum/eth/protocols/diff" "github.com/ethereum/go-ethereum/eth/protocols/eth" "github.com/ethereum/go-ethereum/eth/protocols/snap" @@ -36,8 +38,9 @@ type ethPeerInfo struct { // ethPeer is a wrapper around eth.Peer to maintain a few extra metadata. type ethPeer struct { *eth.Peer - snapExt *snapPeer // Satellite `snap` connection - diffExt *diffPeer + snapExt *snapPeer // Satellite `snap` connection + diffExt *diffPeer + trustExt *trustPeer syncDrop *time.Timer // Connection dropper if `eth` sync progress isn't validated in time snapWait chan struct{} // Notification channel for snap connections @@ -67,6 +70,12 @@ type diffPeerInfo struct { DiffSync bool `json:"diff_sync"` } +// trustPeerInfo represents a short summary of the `trust` sub-protocol metadata known +// about a connected peer. +type trustPeerInfo struct { + Version uint `json:"version"` // Trust protocol version negotiated +} + // snapPeer is a wrapper around snap.Peer to maintain a few extra metadata. type snapPeer struct { *snap.Peer @@ -77,6 +86,11 @@ type diffPeer struct { *diff.Peer } +// trustPeer is a wrapper around trust.Peer to maintain a few extra metadata. +type trustPeer struct { + *trust.Peer +} + // info gathers and returns some `diff` protocol metadata known about a peer. func (p *diffPeer) info() *diffPeerInfo { return &diffPeerInfo{ @@ -91,3 +105,10 @@ func (p *snapPeer) info() *snapPeerInfo { Version: p.Version(), } } + +// info gathers and returns some `trust` protocol metadata known about a peer. +func (p *trustPeer) info() *trustPeerInfo { + return &trustPeerInfo{ + Version: p.Version(), + } +} diff --git a/eth/peerset.go b/eth/peerset.go index 0f5245a05e..b68d0e7783 100644 --- a/eth/peerset.go +++ b/eth/peerset.go @@ -23,10 +23,12 @@ import ( "time" "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core" "github.com/ethereum/go-ethereum/eth/downloader" "github.com/ethereum/go-ethereum/eth/protocols/diff" "github.com/ethereum/go-ethereum/eth/protocols/eth" "github.com/ethereum/go-ethereum/eth/protocols/snap" + "github.com/ethereum/go-ethereum/eth/protocols/trust" "github.com/ethereum/go-ethereum/p2p" ) @@ -53,6 +55,10 @@ var ( // errDiffWithoutEth is returned if a peer attempts to connect only on the // diff protocol without advertising the eth main protocol. errDiffWithoutEth = errors.New("peer connected on diff without compatible eth support") + + // errTrustWithoutEth is returned if a peer attempts to connect only on the + // trust protocol without advertising the eth main protocol. + errTrustWithoutEth = errors.New("peer connected on trust without compatible eth support") ) const ( @@ -73,6 +79,9 @@ type peerSet struct { diffWait map[string]chan *diff.Peer // Peers connected on `eth` waiting for their diff extension diffPend map[string]*diff.Peer // Peers connected on the `diff` protocol, but not yet on `eth` + trustWait map[string]chan *trust.Peer // Peers connected on `eth` waiting for their trust extension + trustPend map[string]*trust.Peer // Peers connected on the `trust` protocol, but not yet on `eth` + lock sync.RWMutex closed bool } @@ -80,11 +89,13 @@ type peerSet struct { // newPeerSet creates a new peer set to track the active participants. func newPeerSet() *peerSet { return &peerSet{ - peers: make(map[string]*ethPeer), - snapWait: make(map[string]chan *snap.Peer), - snapPend: make(map[string]*snap.Peer), - diffWait: make(map[string]chan *diff.Peer), - diffPend: make(map[string]*diff.Peer), + peers: make(map[string]*ethPeer), + snapWait: make(map[string]chan *snap.Peer), + snapPend: make(map[string]*snap.Peer), + diffWait: make(map[string]chan *diff.Peer), + diffPend: make(map[string]*diff.Peer), + trustWait: make(map[string]chan *trust.Peer), + trustPend: make(map[string]*trust.Peer), } } @@ -148,6 +159,40 @@ func (ps *peerSet) registerDiffExtension(peer *diff.Peer) error { return nil } +// registerTrustExtension unblocks an already connected `eth` peer waiting for its +// `trust` extension, or if no such peer exists, tracks the extension for the time +// being until the `eth` main protocol starts looking for it. +func (ps *peerSet) registerTrustExtension(peer *trust.Peer) error { + // Reject the peer if it advertises `trust` without `eth` as `trust` is only a + // satellite protocol meaningful with the chain selection of `eth` + if !peer.RunningCap(eth.ProtocolName, eth.ProtocolVersions) { + return errTrustWithoutEth + } + // If the peer isn't verify node, don't register trust extension into eth protocol. + if !peer.VerifyNode() { + return nil + } + // Ensure nobody can double connect + ps.lock.Lock() + defer ps.lock.Unlock() + + id := peer.ID() + if _, ok := ps.peers[id]; ok { + return errPeerAlreadyRegistered // avoid connections with the same id as existing ones + } + if _, ok := ps.trustPend[id]; ok { + return errPeerAlreadyRegistered // avoid connections with the same id as pending ones + } + // Inject the peer into an `eth` counterpart is available, otherwise save for later + if wait, ok := ps.trustWait[id]; ok { + delete(ps.trustWait, id) + wait <- peer + return nil + } + ps.trustPend[id] = peer + return nil +} + // waitExtensions blocks until all satellite protocols are connected and tracked // by the peerset. func (ps *peerSet) waitSnapExtension(peer *eth.Peer) (*snap.Peer, error) { @@ -234,6 +279,53 @@ func (ps *peerSet) waitDiffExtension(peer *eth.Peer) (*diff.Peer, error) { } } +// waitTrustExtension blocks until all satellite protocols are connected and tracked +// by the peerset. +func (ps *peerSet) waitTrustExtension(peer *eth.Peer) (*trust.Peer, error) { + // If the peer does not support a compatible `trust`, don't wait + if !peer.RunningCap(trust.ProtocolName, trust.ProtocolVersions) { + return nil, nil + } + // If the peer isn't verify node, don't register trust extension into eth protocol. + if !peer.VerifyNode() { + return nil, nil + } + // Ensure nobody can double connect + ps.lock.Lock() + + id := peer.ID() + if _, ok := ps.peers[id]; ok { + ps.lock.Unlock() + return nil, errPeerAlreadyRegistered // avoid connections with the same id as existing ones + } + if _, ok := ps.trustWait[id]; ok { + ps.lock.Unlock() + return nil, errPeerAlreadyRegistered // avoid connections with the same id as pending ones + } + // If `trust` already connected, retrieve the peer from the pending set + if trust, ok := ps.trustPend[id]; ok { + delete(ps.trustPend, id) + + ps.lock.Unlock() + return trust, nil + } + // Otherwise wait for `trust` to connect concurrently + wait := make(chan *trust.Peer) + ps.trustWait[id] = wait + ps.lock.Unlock() + + select { + case peer := <-wait: + return peer, nil + + case <-time.After(extensionWaitTimeout): + ps.lock.Lock() + delete(ps.trustWait, id) + ps.lock.Unlock() + return nil, errPeerWaitTimeout + } +} + func (ps *peerSet) GetDiffPeer(pid string) downloader.IDiffPeer { if p := ps.peer(pid); p != nil && p.diffExt != nil { return p.diffExt @@ -241,9 +333,23 @@ func (ps *peerSet) GetDiffPeer(pid string) downloader.IDiffPeer { return nil } +// GetVerifyPeers returns an array of verify nodes. +func (ps *peerSet) GetVerifyPeers() []core.VerifyPeer { + ps.lock.RLock() + defer ps.lock.RUnlock() + + res := make([]core.VerifyPeer, 0) + for _, p := range ps.peers { + if p.trustExt != nil && p.trustExt.Peer != nil { + res = append(res, p.trustExt.Peer) + } + } + return res +} + // registerPeer injects a new `eth` peer into the working set, or returns an error // if the peer is already known. -func (ps *peerSet) registerPeer(peer *eth.Peer, ext *snap.Peer, diffExt *diff.Peer) error { +func (ps *peerSet) registerPeer(peer *eth.Peer, ext *snap.Peer, diffExt *diff.Peer, trustExt *trust.Peer) error { // Start tracking the new peer ps.lock.Lock() defer ps.lock.Unlock() @@ -265,6 +371,9 @@ func (ps *peerSet) registerPeer(peer *eth.Peer, ext *snap.Peer, diffExt *diff.Pe if diffExt != nil { eth.diffExt = &diffPeer{diffExt} } + if trustExt != nil { + eth.trustExt = &trustPeer{trustExt} + } ps.peers[id] = eth return nil } diff --git a/eth/protocols/diff/protocol.go b/eth/protocols/diff/protocol.go index 4467d0b327..e6bf5b3e14 100644 --- a/eth/protocols/diff/protocol.go +++ b/eth/protocols/diff/protocol.go @@ -92,7 +92,7 @@ func (p *DiffLayersPacket) Unpack() ([]*types.DiffLayer, error) { var diffHash common.Hash hasher.Sum(diffHash[:0]) hasher.Reset() - diff.DiffHash = diffHash + diff.DiffHash.Store(diffHash) } return diffLayers, nil } diff --git a/eth/protocols/trust/discovery.go b/eth/protocols/trust/discovery.go new file mode 100644 index 0000000000..ce38ec5ed9 --- /dev/null +++ b/eth/protocols/trust/discovery.go @@ -0,0 +1,14 @@ +package trust + +import "github.com/ethereum/go-ethereum/rlp" + +// enrEntry is the ENR entry which advertises `trust` protocol on the discovery. +type enrEntry struct { + // Ignore additional fields (for forward compatibility). + Rest []rlp.RawValue `rlp:"tail"` +} + +// ENRKey implements enr.Entry. +func (e enrEntry) ENRKey() string { + return "trust" +} diff --git a/eth/protocols/trust/handler.go b/eth/protocols/trust/handler.go new file mode 100644 index 0000000000..f10aff5178 --- /dev/null +++ b/eth/protocols/trust/handler.go @@ -0,0 +1,157 @@ +package trust + +import ( + "fmt" + "time" + + "github.com/ethereum/go-ethereum/core" + "github.com/ethereum/go-ethereum/metrics" + "github.com/ethereum/go-ethereum/p2p" + "github.com/ethereum/go-ethereum/p2p/enode" + "github.com/ethereum/go-ethereum/p2p/enr" +) + +// Handler is a callback to invoke from an outside runner after the boilerplate +// exchanges have passed. +type Handler func(peer *Peer) error + +type Backend interface { + // Chain retrieves the blockchain object to serve data. + Chain() *core.BlockChain + + // RunPeer is invoked when a peer joins on the `eth` protocol. The handler + // should do any peer maintenance work, handshakes and validations. If all + // is passed, control should be given back to the `handler` to process the + // inbound messages going forward. + RunPeer(peer *Peer, handler Handler) error + + PeerInfo(id enode.ID) interface{} + + Handle(peer *Peer, packet Packet) error +} + +// MakeProtocols constructs the P2P protocol definitions for `trust`. +func MakeProtocols(backend Backend, dnsdisc enode.Iterator) []p2p.Protocol { + // Filter the discovery iterator for nodes advertising trust support. + dnsdisc = enode.Filter(dnsdisc, func(n *enode.Node) bool { + var trust enrEntry + return n.Load(&trust) == nil + }) + + protocols := make([]p2p.Protocol, len(ProtocolVersions)) + for i, version := range ProtocolVersions { + version := version // Closure + + protocols[i] = p2p.Protocol{ + Name: ProtocolName, + Version: version, + Length: protocolLengths[version], + Run: func(p *p2p.Peer, rw p2p.MsgReadWriter) error { + return backend.RunPeer(NewPeer(version, p, rw), func(peer *Peer) error { + defer peer.Close() + return Handle(backend, peer) + }) + }, + NodeInfo: func() interface{} { + return nodeInfo(backend.Chain()) + }, + PeerInfo: func(id enode.ID) interface{} { + return backend.PeerInfo(id) + }, + Attributes: []enr.Entry{&enrEntry{}}, + DialCandidates: dnsdisc, + } + } + return protocols +} + +// Handle is the callback invoked to manage the life cycle of a `trust` peer. +// When this function terminates, the peer is disconnected. +func Handle(backend Backend, peer *Peer) error { + for { + if err := handleMessage(backend, peer); err != nil { + peer.Log().Debug("Message handling failed in `trust`", "err", err) + return err + } + } +} + +// handleMessage is invoked whenever an inbound message is received from a +// remote peer on the `diff` protocol. The remote connection is torn down upon +// returning any error. +func handleMessage(backend Backend, peer *Peer) error { + // Read the next message from the remote peer, and ensure it's fully consumed + msg, err := peer.rw.ReadMsg() + if err != nil { + return err + } + if msg.Size > maxMessageSize { + return fmt.Errorf("%w: %v > %v", errMsgTooLarge, msg.Size, maxMessageSize) + } + defer msg.Discard() + + // Track the amount of time it takes to serve the request and run the handler + if metrics.Enabled { + h := fmt.Sprintf("%s/%s/%d/%#02x", p2p.HandleHistName, ProtocolName, peer.Version(), msg.Code) + defer func(start time.Time) { + sampler := func() metrics.Sample { + return metrics.ResettingSample( + metrics.NewExpDecaySample(1028, 0.015), + ) + } + metrics.GetOrRegisterHistogramLazy(h, nil, sampler).Update(time.Since(start).Microseconds()) + }(time.Now()) + } + // Handle the message depending on its contents + switch { + case msg.Code == RequestRootMsg: + return handleRootRequest(backend, msg, peer) + + case msg.Code == RespondRootMsg: + return handleRootResponse(backend, msg, peer) + + default: + return fmt.Errorf("%w: %v", errInvalidMsgCode, msg.Code) + } +} + +type Decoder interface { + Decode(val interface{}) error + Time() time.Time +} + +func handleRootRequest(backend Backend, msg Decoder, peer *Peer) error { + req := new(RootRequestPacket) + if err := msg.Decode(req); err != nil { + return fmt.Errorf("%w: message %v: %v", errDecode, msg, err) + } + + res := backend.Chain().GetVerifyResult(req.BlockNumber, req.BlockHash, req.DiffHash) + return p2p.Send(peer.rw, RespondRootMsg, RootResponsePacket{ + RequestId: req.RequestId, + Status: res.Status, + BlockNumber: req.BlockNumber, + BlockHash: req.BlockHash, + Root: res.Root, + Extra: defaultExtra, + }) +} + +func handleRootResponse(backend Backend, msg Decoder, peer *Peer) error { + res := new(RootResponsePacket) + if err := msg.Decode(res); err != nil { + return fmt.Errorf("%w: message %v: %v", errDecode, msg, err) + } + + requestTracker.Fulfil(peer.id, peer.version, RespondRootMsg, res.RequestId) + return backend.Handle(peer, res) +} + +// NodeInfo represents a short summary of the `trust` sub-protocol metadata +// known about the host peer. +type NodeInfo struct{} + +// nodeInfo retrieves some `trust` protocol metadata about the running host node. +func nodeInfo(chain *core.BlockChain) *NodeInfo { + return &NodeInfo{} +} diff --git a/eth/protocols/trust/handler_test.go b/eth/protocols/trust/handler_test.go new file mode 100644 index 0000000000..8e9ef6ad58 --- /dev/null +++ b/eth/protocols/trust/handler_test.go @@ -0,0 +1,271 @@ +package trust + +import ( + "math/big" + "testing" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/consensus/clique" + "github.com/ethereum/go-ethereum/core" + "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/core/vm" + "github.com/ethereum/go-ethereum/crypto" + "github.com/ethereum/go-ethereum/ethdb" + "github.com/ethereum/go-ethereum/p2p" + "github.com/ethereum/go-ethereum/p2p/enode" + "github.com/ethereum/go-ethereum/params" +) + +var ( + // testKey is a private key to use for funding a tester account. + testKey, _ = crypto.HexToECDSA("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291") + + // testAddr is the Ethereum address of the tester account. + testAddr = crypto.PubkeyToAddress(testKey.PublicKey) +) + +// testBackend is a mock implementation of the live Ethereum message handler. Its +// purpose is to allow testing the request/reply workflows and wire serialization +// in the `eth` protocol without actually doing any data processing. +type testBackend struct { + db ethdb.Database + chain *core.BlockChain + txpool *core.TxPool +} + +// newTestBackend creates an empty chain and wraps it into a mock backend. +func newTestBackend(blocks int) *testBackend { + return newTestBackendWithGenerator(blocks) +} + +// newTestBackend creates a chain with a number of explicitly defined blocks and +// wraps it into a mock backend. +func newTestBackendWithGenerator(blocks int) *testBackend { + signer := types.HomesteadSigner{} + db := rawdb.NewMemoryDatabase() + engine := clique.New(params.AllCliqueProtocolChanges.Clique, db) + genspec := &core.Genesis{ + //Config: params.TestChainConfig, + ExtraData: make([]byte, 32+common.AddressLength+65), + Alloc: core.GenesisAlloc{testAddr: {Balance: big.NewInt(100000000000000000)}}, + BaseFee: big.NewInt(0), + } + copy(genspec.ExtraData[32:], testAddr[:]) + genesis := genspec.MustCommit(db) + + chain, _ := core.NewBlockChain(db, nil, params.AllCliqueProtocolChanges, engine, vm.Config{}, nil, nil) + generator := func(i int, block *core.BlockGen) { + // The chain maker doesn't have access to a chain, so the difficulty will be + // lets unset (nil). Set it here to the correct value. + // block.SetCoinbase(testAddr) + block.SetDifficulty(big.NewInt(2)) + + // We want to simulate an empty middle block, having the same state as the + // first one. The last is needs a state change again to force a reorg. + tx, err := types.SignTx(types.NewTransaction(block.TxNonce(testAddr), common.Address{0x01}, big.NewInt(1), params.TxGas, nil, nil), signer, testKey) + if err != nil { + panic(err) + } + block.AddTxWithChain(chain, tx) + } + + bs, _ := core.GenerateChain(params.AllCliqueProtocolChanges, genesis, engine, db, blocks, generator) + for i, block := range bs { + header := block.Header() + if i > 0 { + header.ParentHash = bs[i-1].Hash() + } + header.Extra = make([]byte, 32+65) + header.Difficulty = big.NewInt(2) + + sig, _ := crypto.Sign(clique.SealHash(header).Bytes(), testKey) + copy(header.Extra[len(header.Extra)-65:], sig) + bs[i] = block.WithSeal(header) + } + + if _, err := chain.InsertChain(bs); err != nil { + panic(err) + } + + txconfig := core.DefaultTxPoolConfig + txconfig.Journal = "" // Don't litter the disk with test journals + + return &testBackend{ + db: db, + chain: chain, + txpool: core.NewTxPool(txconfig, params.AllCliqueProtocolChanges, chain), + } +} + +// close tears down the transaction pool and chain behind the mock backend. +func (b *testBackend) close() { + b.txpool.Stop() + b.chain.Stop() +} + +func (b *testBackend) Chain() *core.BlockChain { return b.chain } + +func (b *testBackend) RunPeer(peer *Peer, handler Handler) error { + // Normally the backend would do peer mainentance and handshakes. All that + // is omitted and we will just give control back to the handler. + return handler(peer) +} +func (b *testBackend) PeerInfo(enode.ID) interface{} { panic("not implemented") } + +func (b *testBackend) Handle(*Peer, Packet) error { + panic("data processing tests should be done in the handler package") +} + +func TestRequestRoot(t *testing.T) { testRequestRoot(t, Trust1) } + +func testRequestRoot(t *testing.T, protocol uint) { + t.Parallel() + + blockNum := 1032 // The latest 1024 blocks' DiffLayer will be cached. + backend := newTestBackend(blockNum) + defer backend.close() + + peer, _ := newTestPeer("peer", protocol, backend) + defer peer.close() + + pairs := []struct { + req RootRequestPacket + res RootResponsePacket + }{ + { + req: RootRequestPacket{ + RequestId: 1, + BlockNumber: 1, + }, + res: RootResponsePacket{ + RequestId: 1, + Status: types.StatusPartiallyVerified, + BlockNumber: 1, + Extra: defaultExtra, + }, + }, + { + req: RootRequestPacket{ + RequestId: 2, + BlockNumber: 128, + }, + res: RootResponsePacket{ + RequestId: 2, + Status: types.StatusFullVerified, + BlockNumber: 128, + Extra: defaultExtra, + }, + }, + { + req: RootRequestPacket{ + RequestId: 3, + BlockNumber: 128, + BlockHash: types.EmptyRootHash, + DiffHash: types.EmptyRootHash, + }, + res: RootResponsePacket{ + RequestId: 3, + Status: types.StatusImpossibleFork, + BlockNumber: 128, + BlockHash: types.EmptyRootHash, + Root: common.Hash{}, + Extra: defaultExtra, + }, + }, + { + req: RootRequestPacket{ + RequestId: 4, + BlockNumber: 128, + DiffHash: types.EmptyRootHash, + }, + res: RootResponsePacket{ + RequestId: 4, + Status: types.StatusDiffHashMismatch, + BlockNumber: 128, + Root: common.Hash{}, + Extra: defaultExtra, + }, + }, + { + req: RootRequestPacket{ + RequestId: 5, + BlockNumber: 1024, + }, + res: RootResponsePacket{ + RequestId: 5, + Status: types.StatusFullVerified, + BlockNumber: 1024, + Extra: defaultExtra, + }, + }, + { + req: RootRequestPacket{ + RequestId: 6, + BlockNumber: 1024, + BlockHash: types.EmptyRootHash, + DiffHash: types.EmptyRootHash, + }, + res: RootResponsePacket{ + RequestId: 6, + Status: types.StatusPossibleFork, + BlockNumber: 1024, + BlockHash: types.EmptyRootHash, + Root: common.Hash{}, + Extra: defaultExtra, + }, + }, + { + req: RootRequestPacket{ + RequestId: 7, + BlockNumber: 1033, + BlockHash: types.EmptyRootHash, + DiffHash: types.EmptyRootHash, + }, + res: RootResponsePacket{ + RequestId: 7, + Status: types.StatusBlockNewer, + BlockNumber: 1033, + BlockHash: types.EmptyRootHash, + Root: common.Hash{}, + Extra: defaultExtra, + }, + }, + { + req: RootRequestPacket{ + RequestId: 8, + BlockNumber: 1044, + BlockHash: types.EmptyRootHash, + DiffHash: types.EmptyRootHash, + }, + res: RootResponsePacket{ + RequestId: 8, + Status: types.StatusBlockTooNew, + BlockNumber: 1044, + BlockHash: types.EmptyRootHash, + Root: common.Hash{}, + Extra: defaultExtra, + }, + }, + } + + for idx, pair := range pairs { + header := backend.Chain().GetHeaderByNumber(pair.req.BlockNumber) + if header != nil { + if pair.res.Status.Code&0xFF00 == types.StatusVerified.Code { + pair.req.BlockHash = header.Hash() + pair.req.DiffHash, _ = core.CalculateDiffHash(backend.Chain().GetTrustedDiffLayer(header.Hash())) + pair.res.BlockHash = pair.req.BlockHash + pair.res.Root = header.Root + } else if pair.res.Status.Code == types.StatusDiffHashMismatch.Code { + pair.req.BlockHash = header.Hash() + pair.res.BlockHash = pair.req.BlockHash + } + } + + p2p.Send(peer.app, RequestRootMsg, pair.req) + if err := p2p.ExpectMsg(peer.app, RespondRootMsg, pair.res); err != nil { + t.Errorf("test %d: root response not expected: %v", idx, err) + } + } +} diff --git a/eth/protocols/trust/peer.go b/eth/protocols/trust/peer.go new file mode 100644 index 0000000000..18ba229914 --- /dev/null +++ b/eth/protocols/trust/peer.go @@ -0,0 +1,66 @@ +package trust + +import ( + "math/rand" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/log" + "github.com/ethereum/go-ethereum/p2p" +) + +// Peer is a collection of relevant information we have about a `trust` peer. +type Peer struct { + id string // Unique ID for the peer, cached + + *p2p.Peer // The embedded P2P package peer + rw p2p.MsgReadWriter // Input/output streams for diff + version uint // Protocol version negotiated + logger log.Logger // Contextual logger with the peer id injected +} + +// NewPeer create a wrapper for a network connection and negotiated protocol +// version. +func NewPeer(version uint, p *p2p.Peer, rw p2p.MsgReadWriter) *Peer { + id := p.ID().String() + peer := &Peer{ + id: id, + Peer: p, + rw: rw, + version: version, + logger: log.New("peer", id[:8]), + } + return peer +} + +// ID retrieves the peer's unique identifier. +func (p *Peer) ID() string { + return p.id +} + +// Version retrieves the peer's negoatiated `diff` protocol version. +func (p *Peer) Version() uint { + return p.version +} + +// Log overrides the P2P logget with the higher level one containing only the id. +func (p *Peer) Log() log.Logger { + return p.logger +} + +// Close signals the broadcast goroutine to terminate. Only ever call this if +// you created the peer yourself via NewPeer. Otherwise let whoever created it +// clean it up! +func (p *Peer) Close() { +} + +func (p *Peer) RequestRoot(blockNumber uint64, blockHash common.Hash, diffHash common.Hash) error { + id := rand.Uint64() + + requestTracker.Track(p.id, p.version, RequestRootMsg, RespondRootMsg, id) + return p2p.Send(p.rw, RequestRootMsg, RootRequestPacket{ + RequestId: id, + BlockNumber: blockNumber, + BlockHash: blockHash, + DiffHash: diffHash, + }) +} diff --git a/eth/protocols/trust/peer_test.go b/eth/protocols/trust/peer_test.go new file mode 100644 index 0000000000..ab229a1b32 --- /dev/null +++ b/eth/protocols/trust/peer_test.go @@ -0,0 +1,42 @@ +package trust + +import ( + "math/rand" + + "github.com/ethereum/go-ethereum/p2p" + "github.com/ethereum/go-ethereum/p2p/enode" +) + +// testPeer is a simulated peer to allow testing direct network calls. +type testPeer struct { + *Peer + + net p2p.MsgReadWriter // Network layer reader/writer to simulate remote messaging + app *p2p.MsgPipeRW // Application layer reader/writer to simulate the local side +} + +// newTestPeer creates a new peer registered at the given data backend. +func newTestPeer(name string, version uint, backend Backend) (*testPeer, <-chan error) { + // Create a message pipe to communicate through + app, net := p2p.MsgPipe() + + // Start the peer on a new thread + var id enode.ID + rand.Read(id[:]) + + peer := NewPeer(version, p2p.NewPeer(id, name, nil), net) + errc := make(chan error, 1) + go func() { + errc <- backend.RunPeer(peer, func(peer *Peer) error { + return Handle(backend, peer) + }) + }() + return &testPeer{app: app, net: net, Peer: peer}, errc +} + +// close terminates the local side of the peer, notifying the remote protocol +// manager of termination. +func (p *testPeer) close() { + p.Peer.Close() + p.app.Close() +} diff --git a/eth/protocols/trust/protocol.go b/eth/protocols/trust/protocol.go new file mode 100644 index 0000000000..e4f98dd324 --- /dev/null +++ b/eth/protocols/trust/protocol.go @@ -0,0 +1,71 @@ +package trust + +import ( + "errors" + + "github.com/ethereum/go-ethereum/core/types" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/rlp" +) + +// Constants to match up protocol versions and messages +const ( + Trust1 = 1 +) + +// ProtocolName is the official short name of the `trust` protocol used during +// devp2p capability negotiation. +const ProtocolName = "trust" + +// ProtocolVersions are the supported versions of the `trust` protocol (first +// is primary). +var ProtocolVersions = []uint{Trust1} + +// protocolLengths are the number of implemented message corresponding to +// different protocol versions. +var protocolLengths = map[uint]uint64{Trust1: 2} + +// maxMessageSize is the maximum cap on the size of a protocol message. +const maxMessageSize = 10 * 1024 * 1024 + +const ( + RequestRootMsg = 0x00 + RespondRootMsg = 0x01 +) + +var defaultExtra = []byte{0x00} + +var ( + errMsgTooLarge = errors.New("message too long") + errDecode = errors.New("invalid message") + errInvalidMsgCode = errors.New("invalid message code") +) + +// Packet represents a p2p message in the `trust` protocol. +type Packet interface { + Name() string // Name returns a string corresponding to the message type. + Kind() byte // Kind returns the message type. +} + +type RootRequestPacket struct { + RequestId uint64 + BlockNumber uint64 + BlockHash common.Hash + DiffHash common.Hash +} + +type RootResponsePacket struct { + RequestId uint64 + Status types.VerifyStatus + BlockNumber uint64 + BlockHash common.Hash + Root common.Hash + Extra rlp.RawValue // for extension +} + +func (*RootRequestPacket) Name() string { return "RequestRoot" } +func (*RootRequestPacket) Kind() byte { return RequestRootMsg } + +func (*RootResponsePacket) Name() string { return "RootResponse" } +func (*RootResponsePacket) Kind() byte { return RespondRootMsg } diff --git a/eth/protocols/trust/tracker.go b/eth/protocols/trust/tracker.go new file mode 100644 index 0000000000..ab492b3fb8 --- /dev/null +++ b/eth/protocols/trust/tracker.go @@ -0,0 +1,10 @@ +package trust + +import ( + "time" + + "github.com/ethereum/go-ethereum/p2p/tracker" +) + +// requestTracker is a singleton tracker for request times. +var requestTracker = tracker.New(ProtocolName, time.Minute) diff --git a/eth/sync.go b/eth/sync.go index b8ac67d3b2..22e2b293fe 100644 --- a/eth/sync.go +++ b/eth/sync.go @@ -189,11 +189,15 @@ func (cs *chainSyncer) modeAndLocalHead() (downloader.SyncMode, *big.Int) { // snap sync pivot, check if we should reenable if pivot := rawdb.ReadLastPivotNumber(cs.handler.database); pivot != nil { if head := cs.handler.chain.CurrentBlock(); head.NumberU64() < *pivot { + if rawdb.ReadAncientType(cs.handler.database) == rawdb.PruneFreezerType { + log.Crit("Current rewound to before the fast sync pivot, can't enable pruneancient mode", "current block number", head.NumberU64(), "pivot", *pivot) + } block := cs.handler.chain.CurrentFastBlock() td := cs.handler.chain.GetTd(block.Hash(), block.NumberU64()) return downloader.SnapSync, td } } + // Nope, we're really full syncing head := cs.handler.chain.CurrentBlock() td := cs.handler.chain.GetTd(head.Hash(), head.NumberU64()) diff --git a/ethclient/ethclient.go b/ethclient/ethclient.go index 951b65940a..db388a3d9d 100644 --- a/ethclient/ethclient.go +++ b/ethclient/ethclient.go @@ -27,6 +27,7 @@ import ( "github.com/ethereum/go-ethereum" "github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common/hexutil" + "github.com/ethereum/go-ethereum/core" "github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/rpc" ) @@ -200,6 +201,12 @@ func (ec *Client) GetDiffAccountsWithScope(ctx context.Context, number *big.Int, return &result, err } +func (ec *Client) GetRootByDiffHash(ctx context.Context, blockNr *big.Int, blockHash common.Hash, diffHash common.Hash) (*core.VerifyResult, error) { + var result core.VerifyResult + err := ec.c.CallContext(ctx, &result, "eth_getRootByDiffHash", toBlockNumArg(blockNr), blockHash, diffHash) + return &result, err +} + type rpcTransaction struct { tx *types.Transaction txExtraInfo diff --git a/go.mod b/go.mod index 918a1c20ac..a6a3acb727 100644 --- a/go.mod +++ b/go.mod @@ -8,6 +8,7 @@ require ( github.com/Azure/go-autorest/autorest/adal v0.8.0 // indirect github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6 // indirect github.com/VictoriaMetrics/fastcache v1.6.0 + github.com/VividCortex/gohistogram v1.0.0 // indirect github.com/aws/aws-sdk-go-v2 v1.2.0 github.com/aws/aws-sdk-go-v2/config v1.1.1 github.com/aws/aws-sdk-go-v2/credentials v1.1.1 @@ -19,8 +20,9 @@ require ( github.com/davecgh/go-spew v1.1.1 github.com/deckarep/golang-set v1.8.0 github.com/deepmap/oapi-codegen v1.8.2 // indirect - github.com/docker/docker v1.4.2-0.20180625184442-8e610b2b55bf - github.com/dop251/goja v0.0.0-20211011172007-d99e4b8cbf48 + github.com/dlclark/regexp2 v1.2.0 // indirect + github.com/docker/docker v1.6.1 + github.com/dop251/goja v0.0.0-20200721192441-a695b0cdd498 github.com/edsrzf/mmap-go v1.0.0 github.com/etcd-io/bbolt v1.3.3 // indirect github.com/fatih/color v1.7.0 @@ -28,6 +30,7 @@ require ( github.com/fortytw2/leaktest v1.3.0 // indirect github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff github.com/go-ole/go-ole v1.2.1 // indirect + github.com/go-sourcemap/sourcemap v2.1.3+incompatible // indirect github.com/go-stack/stack v1.8.0 github.com/golang/protobuf v1.4.3 github.com/golang/snappy v0.0.4 @@ -49,13 +52,13 @@ require ( github.com/julienschmidt/httprouter v1.2.0 github.com/karalabe/usb v0.0.2 github.com/kylelemons/godebug v1.1.0 // indirect + github.com/magiconair/properties v1.8.6 // indirect github.com/mattn/go-colorable v0.1.8 github.com/mattn/go-isatty v0.0.12 github.com/naoina/go-stringutil v0.1.0 // indirect github.com/naoina/toml v0.1.2-0.20170918210437-9fafd6967416 github.com/olekukonko/tablewriter v0.0.5 github.com/panjf2000/ants/v2 v2.4.5 - github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c github.com/peterh/liner v1.1.1-0.20190123174540-a2c9a5303de7 github.com/prometheus/tsdb v0.7.1 github.com/rcrowley/go-metrics v0.0.0-20190826022208-cac0b30c2563 // indirect @@ -64,10 +67,10 @@ require ( github.com/shirou/gopsutil v3.21.4-0.20210419000835-c7a38de76ee5+incompatible github.com/status-im/keycard-go v0.0.0-20190316090335-8537d3370df4 github.com/stretchr/testify v1.7.0 + github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 github.com/tendermint/go-amino v0.14.1 github.com/tendermint/iavl v0.12.0 github.com/tendermint/tendermint v0.31.11 - github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 github.com/tklauser/go-sysconf v0.3.5 // indirect github.com/tyler-smith/go-bip39 v1.0.1-0.20181017060643-dbb3b84ba2ef go.etcd.io/bbolt v1.3.5 // indirect @@ -80,5 +83,9 @@ require ( gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce gopkg.in/olebedev/go-duktape.v3 v3.0.0-20200619000410-60c24ae608a6 gopkg.in/urfave/cli.v1 v1.20.0 - gotest.tools v2.2.0+incompatible // indirect +) + +replace ( + github.com/gogo/protobuf v1.1.1 => github.com/gogo/protobuf v1.3.2 + github.com/gogo/protobuf v1.3.1 => github.com/gogo/protobuf v1.3.2 ) diff --git a/go.sum b/go.sum index 5ac0ad9124..5ef6be435d 100644 --- a/go.sum +++ b/go.sum @@ -7,24 +7,16 @@ cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxK cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= -cloud.google.com/go v0.51.0 h1:PvKAVQWCtlGUSlZkGW3QLelKaWq7KYv/MW1EboG8bfM= cloud.google.com/go v0.51.0/go.mod h1:hWtGJ6gnXH+KgDv+V0zFGDvpi07n3z8ZNj3T1RW0Gcw= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= -cloud.google.com/go/bigquery v1.3.0 h1:sAbMqjY1PEQKZBWfbu6Y6bsupJ9c4QdHnzg/VvYTLcE= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= -cloud.google.com/go/bigtable v1.2.0 h1:F4cCmA4nuV84V5zYQ3MKY+M1Cw1avHDuf3S/LcZPA9c= cloud.google.com/go/bigtable v1.2.0/go.mod h1:JcVAOl45lrTmQfLj7T6TxyMzIN/3FGGcFm+2xVAli2o= -cloud.google.com/go/datastore v1.0.0 h1:Kt+gOPPp2LEPWp8CSfxhsM8ik9CcyE/gYu+0r+RnZvM= cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= -cloud.google.com/go/pubsub v1.1.0 h1:9/vpR43S4aJaROxqQHQ3nH9lfyKKV0dC3vOmnw8ebQQ= cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= -cloud.google.com/go/storage v1.5.0 h1:RPUcBvDeYgQFMfQu1eBMq6piD1SXmLH+vK3qjewZPus= cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= -collectd.org v0.3.0 h1:iNBHGw1VvPJxH2B6RiFWFZ+vsjo1lCdRszBeOuwGi00= collectd.org v0.3.0/go.mod h1:A/8DzQBkF6abtvrT2j/AU/4tiBgJWYyh0y/oB/4MlWE= -dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9 h1:VpgP7xuJadIUuKccphEpTJnWhS2jkQyMt6Y7pJCD7fY= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= github.com/Azure/azure-pipeline-go v0.2.1/go.mod h1:UGSo8XybXnIGZ3epmeBw7Jdz+HiUVpqIlpz/HKHylF4= github.com/Azure/azure-pipeline-go v0.2.2 h1:6oiIS9yaG6XCCzhgAgKFfIWyo4LLCiDhZot6ltoThhY= @@ -47,31 +39,23 @@ github.com/Azure/go-autorest/logger v0.1.0 h1:ruG4BSDXONFRrZZJ2GUXDiUyVpayPmb1Gn github.com/Azure/go-autorest/logger v0.1.0/go.mod h1:oExouG+K6PryycPJfVSxi/koC6LSNgds39diKLz7Vrc= github.com/Azure/go-autorest/tracing v0.5.0 h1:TRn4WjSnkcSy5AEG3pnbtFSwNtwzjr4VYyQflFE619k= github.com/Azure/go-autorest/tracing v0.5.0/go.mod h1:r/s2XiOKccPW3HrqB+W0TQzfbtp2fGCgRFtBroKn4Dk= -github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802 h1:1BDTz0u9nC3//pOCMdNH+CiXJVYJh5UQNCOBG7jbELc= github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= -github.com/DATA-DOG/go-sqlmock v1.3.3 h1:CWUqKXe0s8A2z6qCgkP4Kru7wC11YoAnoupUKFDnH08= github.com/DATA-DOG/go-sqlmock v1.3.3/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM= -github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE= github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6 h1:fLjPD/aNc3UIOA6tDi6QXUemppXK3P9BI7mr2hd6gx8= github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6/go.mod h1:3eOhrUMpNV+6aFIbp5/iudMxNCF27Vw2OZgy4xEx0Fg= github.com/VictoriaMetrics/fastcache v1.6.0 h1:C/3Oi3EiBCqufydp1neRZkqcwmEiuRT9c3fqvvgKm5o= github.com/VictoriaMetrics/fastcache v1.6.0/go.mod h1:0qHz5QP0GMX4pfmMA/zt5RgfNuXJrTP0zS7DqpHGGTw= -github.com/aead/siphash v1.0.1 h1:FwHfE/T45KPKYuuSAKyyvE+oPWcaQ+CUmFW0bPlM+kg= +github.com/VividCortex/gohistogram v1.0.0 h1:6+hBz+qvs0JOrrNhhmR7lFxo5sINxBCGXrdtl/UvroE= +github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g= github.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII= -github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af h1:wVe6/Ea46ZMeNkQjjBW6xcqyQA/j5e0D6GytH95g0gQ= github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af/go.mod h1:K08gAheRH3/J6wwsYMMT4xOr94bZjxIelGM0+d/wbFw= -github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc h1:cAKDfWh5VpdgMhJosfJnn5/FoN2SRZ4p7fJNX58YPaU= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= -github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf h1:qet1QNfXsQxTZqLG4oE62mJzwPIB8+Tee4RNCL9ulrY= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156 h1:eMwmnE/GDgah4HI848JfFxHt+iPb26b4zyfspmqY0/8= github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156/go.mod h1:Cb/ax3seSYIx7SuZdm2G2xzfwmv3TPSk2ucNfQESPXM= -github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883 h1:bvNMNQO63//z+xNgfBlViaCIJKLlCJ6/fmUseuG0wVQ= github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8= -github.com/apache/arrow/go/arrow v0.0.0-20191024131854-af6fa24be0db h1:nxAtV4VajJDhKysp2kdcJZsq8Ss1xSA0vZTkVHHJd0E= github.com/apache/arrow/go/arrow v0.0.0-20191024131854-af6fa24be0db/go.mod h1:VTxUBvSJ3s3eHAg65PNgrsn5BtqCRPdmyXh6rAfdxN0= github.com/aws/aws-sdk-go-v2 v1.2.0 h1:BS+UYpbsElC82gB+2E2jiCBg36i8HlubTB/dO/moQ9c= github.com/aws/aws-sdk-go-v2 v1.2.0/go.mod h1:zEQs02YRBw1DjK0PoJv3ygDYOFTre1ejlJWl8FwAuQo= @@ -94,29 +78,19 @@ github.com/aws/smithy-go v1.1.0/go.mod h1:EzMw8dbp/YJL4A5/sbhGddag+NPT7q084agLbB github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= github.com/beorn7/perks v1.0.0 h1:HWo1m869IqiPhD389kmkxeTalrjNbbJTC8LXupb+sl0= github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= -github.com/bmizerany/pat v0.0.0-20170815010413-6226ea591a40 h1:y4B3+GPxKlrigF1ha5FFErxK+sr6sWxQovRMzwMhejo= github.com/bmizerany/pat v0.0.0-20170815010413-6226ea591a40/go.mod h1:8rLXio+WjiTceGBHIoTvn60HIbs7Hm7bcHjyrSqYB9c= -github.com/boltdb/bolt v1.3.1 h1:JQmyP4ZBrce+ZQu0dY660FMfatumYDLun9hBCUVIkF4= github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps= github.com/btcsuite/btcd v0.20.1-beta h1:Ik4hyJqN8Jfyv3S4AGBOmyouMsYE3EdYODkMbQjwPGw= github.com/btcsuite/btcd v0.20.1-beta/go.mod h1:wVuoA8VJLEcwgqHBwHmzLRazpKxTv13Px/pDuV7OomQ= -github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f h1:bAs4lUbRJpnnkd9VhRV3jjAVU7DJVjMaK+IsvSeZvFo= github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f/go.mod h1:TdznJufoqS23FtqVCzL0ZqgP5MqXbb4fg/WgDys70nA= github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d h1:yJzD/yFppdVCf6ApMkVy8cUxV0XrxdP9rVf6D87/Mng= github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d/go.mod h1:+5NJ2+qvTyV9exUAL/rxXi3DcLg2Ts+ymUAY5y4NvMg= -github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd h1:R/opQEbFEy9JGkIguV40SvRY1uliPX8ifOvi6ICsFCw= github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd/go.mod h1:HHNXQzUsZCxOoE+CPiyCTO6x34Zs86zZUiwtpXoGdtg= -github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd h1:qdGvebPBDuYDPGi1WCPjy1tGyMpmDK8IEapSsszn7HE= github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd/go.mod h1:F+uVaaLLH7j4eDXPRvw78tMflu7Ie2bzYOH4Y8rRKBY= -github.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723 h1:ZA/jbKoGcVAnER6pCHPEkGdZOV7U1oLUedErBHCUMs0= github.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg3lh6TiUghc= -github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792 h1:R8vQdOQdZ9Y3SkEwmHoWBmX1DNXhXZqlTpq6s4tyJGc= github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtEyQwv5/p4Mg4C0fgbePVuGr935/5ddU9Z3TmDRY= -github.com/btcsuite/winsvc v1.0.0 h1:J9B4L7e3oqhXOcm+2IuNApwzQec85lE+QaikUcCs+dk= github.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs= -github.com/c-bata/go-prompt v0.2.2 h1:uyKRz6Z6DUyj49QVijyM339UJV9yhbr70gESwbNU3e0= github.com/c-bata/go-prompt v0.2.2/go.mod h1:VzqtzE2ksDBcdln8G7mk2RX9QyGjH+OVqOCSiVIqS34= -github.com/census-instrumentation/opencensus-proto v0.2.1 h1:glEXhBS5PSLLv4IXzLA5yPRVX4bilULVyxxbrfOtDAk= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= github.com/cespare/cp v0.1.0 h1:SE+dxFebS7Iik5LK0tsi1k9ZCxEaFX4AjQmoyA+1dJk= github.com/cespare/cp v0.1.0/go.mod h1:SOGHArjBr4JWaSDEVpWpo/hNg6RoKrls6Oh40hiwW+s= @@ -124,27 +98,17 @@ github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY= github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/chzyer/logex v1.1.10 h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= -github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= -github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 h1:q763qf9huN11kDQavWsoZXJNW3xEE4JJyHa5Q25/sd8= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= -github.com/client9/misspell v0.3.4 h1:ta993UF76GwbvJcIo3Y68y/M3WxlpEHPWIGDkJYwzJI= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/cloudflare/cloudflare-go v0.14.0 h1:gFqGlGl/5f9UGXAaKapCGUfaTCgRKKnzu2VvzMZlOFA= github.com/cloudflare/cloudflare-go v0.14.0/go.mod h1:EnwdgGMaFOruiPZRFSgn+TsQ3hQ7C/YWzIGLeu5c304= -github.com/consensys/bavard v0.1.8-0.20210406032232-f3452dc9b572 h1:+R8G1+Ftumd0DaveLgMIjrFPcAS4G8MsVXWXiyZL5BY= github.com/consensys/bavard v0.1.8-0.20210406032232-f3452dc9b572/go.mod h1:Bpd0/3mZuaj6Sj+PqrmIquiOKy397AKGThQPaGzNXAQ= github.com/consensys/gnark-crypto v0.4.1-0.20210426202927-39ac3d4b3f1f h1:C43yEtQ6NIf4ftFXD/V55gnGFgPbMQobd//YlnLjUJ8= github.com/consensys/gnark-crypto v0.4.1-0.20210426202927-39ac3d4b3f1f/go.mod h1:815PAHg3wvysy0SyIqanF8gZ0Y1wjk/hrDHD/iT88+Q= -github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d h1:U+s90UTSYgptZMwQh2aRr3LuazLJIa+Pg3Kc1ylSYVY= github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= -github.com/creack/pty v1.1.9 h1:uDmaGzcdjhF4i/plgjmEsriH11Y0o7RKapEf/LDaM3w= -github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= -github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c h1:/ovYnF02fwL0kvspmy9AuyKg1JhdTRUgPw4nUxd9oZM= github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c/go.mod h1:GyV+0YP4qX0UQ7r2MoYZ+AvYDp12OF5yg4q8rGnyNh4= -github.com/dave/jennifer v1.2.0 h1:S15ZkFMRoJ36mGAQgWL1tnr0NQJh9rZ8qatseX/VbBc= github.com/dave/jennifer v1.2.0/go.mod h1:fIb+770HOpJ2fmN9EPPKOqm1vMGhB+TwXKMZhrIygKg= github.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -157,25 +121,18 @@ github.com/deepmap/oapi-codegen v1.8.2 h1:SegyeYGcdi0jLLrpbCMoJxnUUn8GBXHsvr4rbz github.com/deepmap/oapi-codegen v1.8.2/go.mod h1:YLgSKSDv/bZQB7N4ws6luhozi3cEdRktEqrX88CvjIw= github.com/dgrijalva/jwt-go v3.2.0+incompatible h1:7qlOGliEKZXTDg6OTjfoBKDXWrumCAMpl/TFQ4/5kLM= github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= -github.com/dgryski/go-bitstream v0.0.0-20180413035011-3522498ce2c8 h1:akOQj8IVgoeFfBTzGOEQakCYshWD6RNo1M5pivFXt70= github.com/dgryski/go-bitstream v0.0.0-20180413035011-3522498ce2c8/go.mod h1:VMaSuZ+SZcx/wljOQKvp5srsbCiKDEb6K2wC4+PiBmQ= -github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954 h1:RMLoZVzv4GliuWafOuPuQDKSm1SJph7uCRnnS61JAn4= github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= -github.com/dlclark/regexp2 v1.4.1-0.20201116162257-a2a8dda75c91 h1:Izz0+t1Z5nI16/II7vuEo/nHjodOg0p7+OiDpjX5t1E= -github.com/dlclark/regexp2 v1.4.1-0.20201116162257-a2a8dda75c91/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc= -github.com/docker/docker v1.4.2-0.20180625184442-8e610b2b55bf h1:sh8rkQZavChcmakYiSlqu2425CHyFXLZZnvm7PDpU8M= -github.com/docker/docker v1.4.2-0.20180625184442-8e610b2b55bf/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= -github.com/dop251/goja v0.0.0-20211011172007-d99e4b8cbf48 h1:iZOop7pqsg+56twTopWgwCGxdB5SI2yDO8Ti7eTRliQ= -github.com/dop251/goja v0.0.0-20211011172007-d99e4b8cbf48/go.mod h1:R9ET47fwRVRPZnOGvHxxhuZcbrMCuiqOz3Rlrh4KSnk= -github.com/dop251/goja_nodejs v0.0.0-20210225215109-d91c329300e7 h1:tYwu/z8Y0NkkzGEh3z21mSWggMg4LwLRFucLS7TjARg= -github.com/dop251/goja_nodejs v0.0.0-20210225215109-d91c329300e7/go.mod h1:hn7BA7c8pLvoGndExHudxTDKZ84Pyvv+90pbBjbTz0Y= -github.com/eclipse/paho.mqtt.golang v1.2.0 h1:1F8mhG9+aO5/xpdtFkW4SxOJB67ukuDC3t2y2qayIX0= +github.com/dlclark/regexp2 v1.2.0 h1:8sAhBGEM0dRWogWqWyQeIJnxjWO6oIjl8FKqREDsGfk= +github.com/dlclark/regexp2 v1.2.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc= +github.com/docker/docker v1.6.1 h1:4xYASHy5cScPkLD7PO0uTmnVc860m9NarPN1X8zeMe8= +github.com/docker/docker v1.6.1/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= +github.com/dop251/goja v0.0.0-20200721192441-a695b0cdd498 h1:Y9vTBSsV4hSwPSj4bacAU/eSnV3dAxVpepaghAdhGoQ= +github.com/dop251/goja v0.0.0-20200721192441-a695b0cdd498/go.mod h1:Mw6PkjjMXWbTj+nnj4s3QPXq1jaT0s5pC0iFD4+BOAA= github.com/eclipse/paho.mqtt.golang v1.2.0/go.mod h1:H9keYFcgq3Qr5OUJm/JZI/i6U7joQ8SYLhZwfeOo6Ts= github.com/edsrzf/mmap-go v1.0.0 h1:CEBF7HpRnUCSJgGUb5h1Gm7e3VkmVDrR8lvWVLtrOFw= github.com/edsrzf/mmap-go v1.0.0/go.mod h1:YO35OhQPt3KJa3ryjFM5Bs14WD66h8eGKpfaBNrHW5M= -github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473 h1:4cmBvAEBNJaGARUEs3/suWRyfyBfhf7I60WBZq+bv2w= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= -github.com/envoyproxy/protoc-gen-validate v0.1.0 h1:EQciDnbrYxy13PgWoY8AqoxGiPrpgBZ1R8UNe3ddc+A= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/etcd-io/bbolt v1.3.3 h1:gSJmxrs37LgTqR/oyJBWok6k6SvXEUerFTbltIhXkBM= github.com/etcd-io/bbolt v1.3.3/go.mod h1:ZF2nL25h33cCyBtcyWeZ2/I3HQOfTP+0PIEvHjkjCrw= @@ -183,7 +140,6 @@ github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= github.com/fjl/memsize v0.0.0-20190710130421-bcb5799ab5e5 h1:FtmdgXiUlNeRsoNMFlKLDt+S+6hbjVMEW6RGQ7aUf7c= github.com/fjl/memsize v0.0.0-20190710130421-bcb5799ab5e5/go.mod h1:VvhXpOYNQvB+uIk2RvXzuaQtkQJzzIx6lSBe1xv7hi0= -github.com/fogleman/gg v1.2.1-0.20190220221249-0403632d5b90 h1:WXb3TSNmHp2vHoCroCIB1foO/yQ36swABL8aOVeDpgg= github.com/fogleman/gg v1.2.1-0.20190220221249-0403632d5b90/go.mod h1:R/bRT+9gY/C5z7JzPU0zXsXHKM4/ayA+zqcVNZzPa1k= github.com/fortytw2/leaktest v1.3.0 h1:u8491cBMTQ8ft8aeV+adlcytMZylmA5nnwwkRZjI8vw= github.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHquHwclZch5g= @@ -193,19 +149,12 @@ github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4 github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff h1:tY80oXqGNY4FhTFhk+o9oFHGINQ/+vhlm8HFzi6znCI= github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff/go.mod h1:x7DCsMOv1taUwEWCzT4cmDeAkigA5/QCwUodaVOe8Ww= github.com/getkin/kin-openapi v0.53.0/go.mod h1:7Yn5whZr5kJi6t+kShccXS8ae1APpYTW6yheSwk8Yi4= -github.com/getkin/kin-openapi v0.61.0 h1:6awGqF5nG5zkVpMsAih1QH4VgzS8phTxECUWIFo7zko= github.com/getkin/kin-openapi v0.61.0/go.mod h1:7Yn5whZr5kJi6t+kShccXS8ae1APpYTW6yheSwk8Yi4= -github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= -github.com/glycerine/go-unsnap-stream v0.0.0-20180323001048-9f0cb55181dd h1:r04MMPyLHj/QwZuMJ5+7tJcBr1AQjpiAK/rZWRrQT7o= github.com/glycerine/go-unsnap-stream v0.0.0-20180323001048-9f0cb55181dd/go.mod h1:/20jfyN9Y5QPEAprSgKAUr+glWDY39ZiUEAYOEv5dsE= -github.com/glycerine/goconvey v0.0.0-20190410193231-58a59202ab31 h1:gclg6gY70GLy3PbkQ1AERPfmLMMagS60DKF78eWwLn8= github.com/glycerine/goconvey v0.0.0-20190410193231-58a59202ab31/go.mod h1:Ogl1Tioa0aV7gstGFO7KhffUsb9M4ydbEbbxpcEDc24= -github.com/go-chi/chi/v5 v5.0.0 h1:DBPx88FjZJH3FsICfDAfIfnb7XxKIYVGG6lOPlhENAg= github.com/go-chi/chi/v5 v5.0.0/go.mod h1:BBug9lr0cqtdAhsu6R4AAdvufI0/XBzAQSsUqJpoZOs= -github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1 h1:QbL/5oDUmRBzO9/Z7Seo6zf912W/a6Sr4Eu0G/3Jho0= github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= -github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72 h1:b+9H1GAsx5RsjvDFLoS5zkNBzIQMuVKUYQDmxU3N5XE= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/go-kit/kit v0.8.0 h1:Wz+5lgoB0kkuqLEc6NVmwRknTKP6dTGbSqvhZtBI/j0= github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= @@ -214,33 +163,24 @@ github.com/go-logfmt/logfmt v0.4.0 h1:MP4Eh7ZCb31lleYCFuwm0oe4/YGak+5l1vA2NOE80n github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= github.com/go-ole/go-ole v1.2.1 h1:2lOsA72HgjxAuMlKpFiCbHTvu44PIVkZ5hqm3RSdI/E= github.com/go-ole/go-ole v1.2.1/go.mod h1:7FAglXiTm7HKlQRDeOQ6ZNUHidzCWXuZWq/1dTyBNF8= -github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY= github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= -github.com/go-openapi/swag v0.19.5 h1:lTz6Ys4CmqqCQmZPBlbQENR1/GucA2bzYTE12Pw4tFY= github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= github.com/go-sourcemap/sourcemap v2.1.3+incompatible h1:W1iEw64niKVGogNgBN3ePyLFfuisuzeidWPMPWmECqU= github.com/go-sourcemap/sourcemap v2.1.3+incompatible/go.mod h1:F8jJfvm2KbVjc5NqelyYJmf/v5J0dwNLS2mL4sNA1Jg= -github.com/go-sql-driver/mysql v1.4.1 h1:g24URVg0OFbNUTx9qqY1IRZ9D9z3iPyi5zKhQZpNwpA= github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= github.com/go-stack/stack v1.8.0 h1:5SgMzNM5HxrEjV0ww2lTmX6E2Izsfxas4+YHWRs3Lsk= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= -github.com/gofrs/uuid v3.3.0+incompatible h1:8K4tyRfvU1CYPgJsveYFQMhpFd/wXNM7iK6rR7UHz84= github.com/gofrs/uuid v3.3.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM= -github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= -github.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls= -github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o= -github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 h1:DACJavvAHhabrF08vX0COfcOBJRhZ8lUbR+ZWIs0Y5g= +github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= +github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0/go.mod h1:E/TSTwGwJL78qG/PmXZO1EjYhfJinVAhrmmHX6Z8B9k= -github.com/golang/geo v0.0.0-20190916061304-5b978397cfec h1:lJwO/92dFXWeXOZdoGXgptLmNLwynMSHUmU6besqtiw= github.com/golang/geo v0.0.0-20190916061304-5b978397cfec/go.mod h1:QZ0nwyI2jOfgRAoBvP+ab5aRr7c9x7lhGEJrKvBwjWI= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7 h1:5ZkaAPbicIKTF2I64qf5Fh8Aa83Q/dnOafMYV0OMwjA= github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/mock v1.3.1 h1:qGJ6qTW+x6xX/my+8YUVl4WNpX9B7+/l2tRsHGZ7f2s= github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= @@ -257,12 +197,9 @@ github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8l github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM= github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= -github.com/golangci/lint-1 v0.0.0-20181222135242-d2cdd8c08219 h1:utua3L2IbQJmauC5IXdEA547bcoU5dozgQAfc8Onsg4= github.com/golangci/lint-1 v0.0.0-20181222135242-d2cdd8c08219/go.mod h1:/X8TswGSh1pIozq4ZwCfxS0WA5JGXguxk94ar/4c87Y= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/btree v1.0.0 h1:0udJVsspx3VBr5FwtLhQQtuAsVc79tTq0ocGIPAU6qo= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/flatbuffers v1.11.0 h1:O7CEyB8Cb3/DmtxODGtLHcEvpr81Jm5qLg/hsHnxA2A= github.com/google/flatbuffers v1.11.0/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= @@ -273,22 +210,16 @@ github.com/google/go-cmp v0.5.4 h1:L8R9j+yAqZuZjsqh/z+F1NCffTKKLShY6zXTItVIZ8M= github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/gofuzz v1.1.1-0.20200604201612-c04b05f3adfa h1:Q75Upo5UN4JbPFURXZ8nLKYUvF85dyFRop/vQ0Rv+64= github.com/google/gofuzz v1.1.1-0.20200604201612-c04b05f3adfa/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= -github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc h1:DLpL8pWq0v4JYoRpEhDfsJhhJyGKCcQM2WPW2TJs31c= github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/renameio v0.1.0 h1:GOZbcHa3HfsPKPlmyPyN2KEohoMXOhdMbHrvbpl2QaA= github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= github.com/google/uuid v1.1.5 h1:kxhtnfFVi+rYdOALN0B3k9UT86zVJKfBimRaciULW4I= github.com/google/uuid v1.1.5/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= -github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM= github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= -github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= -github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI= github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So= github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc= github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= @@ -304,107 +235,73 @@ github.com/holiman/bloomfilter/v2 v2.0.3 h1:73e0e/V0tCydx14a0SCYS/EWCxgwLZ18CZcZ github.com/holiman/bloomfilter/v2 v2.0.3/go.mod h1:zpoh+gs7qcpqrHr3dB55AMiJwo0iURXE7ZOP9L9hSkA= github.com/holiman/uint256 v1.2.0 h1:gpSYcPLWGv4sG43I2mVLiDZCNDh/EpGjSk8tmtxitHM= github.com/holiman/uint256 v1.2.0/go.mod h1:y4ga/t+u+Xwd7CpDgZESaRcWy0I7XMlTMA25ApIH5Jw= -github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI= github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/huin/goupnp v1.0.2 h1:RfGLP+h3mvisuWEyybxNq5Eft3NWhHLPeUN72kpKZoI= github.com/huin/goupnp v1.0.2/go.mod h1:0dxJBVBHqTMjIUMkESDTNgOOx/Mw5wYIfyFmdzSamkM= -github.com/huin/goutil v0.0.0-20170803182201-1ca381bf3150 h1:vlNjIqmUZ9CMAWsbURYl3a6wZbw7q5RHVvlXTNS/Bs8= github.com/huin/goutil v0.0.0-20170803182201-1ca381bf3150/go.mod h1:PpLOETDnJ0o3iZrZfqZzyLl6l7F3c6L1oWn7OICBi6o= -github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6 h1:UDMh68UUwekSh5iP2OMhRRZJiiBccgV7axzUG8vi56c= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= -github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM= github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= -github.com/influxdata/flux v0.65.1 h1:77BcVUCzvN5HMm8+j9PRBQ4iZcu98Dl4Y9rf+J5vhnc= github.com/influxdata/flux v0.65.1/go.mod h1:J754/zds0vvpfwuq7Gc2wRdVwEodfpCFM7mYlOw2LqY= github.com/influxdata/influxdb v1.8.3 h1:WEypI1BQFTT4teLM+1qkEcvUi0dAvopAI/ir0vAiBg8= github.com/influxdata/influxdb v1.8.3/go.mod h1:JugdFhsvvI8gadxOI6noqNeeBHvWNTbfYGtiAn+2jhI= github.com/influxdata/influxdb-client-go/v2 v2.4.0 h1:HGBfZYStlx3Kqvsv1h2pJixbCl/jhnFtxpKFAv9Tu5k= github.com/influxdata/influxdb-client-go/v2 v2.4.0/go.mod h1:vLNHdxTJkIf2mSLvGrpj8TCcISApPoXkaxP8g9uRlW8= -github.com/influxdata/influxql v1.1.1-0.20200828144457-65d3ef77d385 h1:ED4e5Cc3z5vSN2Tz2GkOHN7vs4Sxe2yds6CXvDnvZFE= github.com/influxdata/influxql v1.1.1-0.20200828144457-65d3ef77d385/go.mod h1:gHp9y86a/pxhjJ+zMjNXiQAA197Xk9wLxaz+fGG+kWk= github.com/influxdata/line-protocol v0.0.0-20180522152040-32c6aa80de5e/go.mod h1:4kt73NQhadE3daL3WhR5EJ/J2ocX0PZzwxQ0gXJ7oFE= github.com/influxdata/line-protocol v0.0.0-20200327222509-2487e7298839/go.mod h1:xaLFMmpvUxqXtVkUJfg9QmT88cDaCJ3ZKgdZ78oO8Qo= github.com/influxdata/line-protocol v0.0.0-20210311194329-9aa0e372d097 h1:vilfsDSy7TDxedi9gyBkMvAirat/oRcL0lFdJBf6tdM= github.com/influxdata/line-protocol v0.0.0-20210311194329-9aa0e372d097/go.mod h1:xaLFMmpvUxqXtVkUJfg9QmT88cDaCJ3ZKgdZ78oO8Qo= -github.com/influxdata/promql/v2 v2.12.0 h1:kXn3p0D7zPw16rOtfDR+wo6aaiH8tSMfhPwONTxrlEc= github.com/influxdata/promql/v2 v2.12.0/go.mod h1:fxOPu+DY0bqCTCECchSRtWfc+0X19ybifQhZoQNF5D8= -github.com/influxdata/roaring v0.4.13-0.20180809181101-fc520f41fab6 h1:UzJnB7VRL4PSkUJHwsyzseGOmrO/r4yA+AuxGJxiZmA= github.com/influxdata/roaring v0.4.13-0.20180809181101-fc520f41fab6/go.mod h1:bSgUQ7q5ZLSO+bKBGqJiCBGAl+9DxyW63zLTujjUlOE= -github.com/influxdata/tdigest v0.0.0-20181121200506-bf2b5ad3c0a9 h1:MHTrDWmQpHq/hkq+7cw9oYAt2PqUw52TZazRA0N7PGE= github.com/influxdata/tdigest v0.0.0-20181121200506-bf2b5ad3c0a9/go.mod h1:Js0mqiSBE6Ffsg94weZZ2c+v/ciT8QRHFOap7EKDrR0= -github.com/influxdata/usage-client v0.0.0-20160829180054-6d3895376368 h1:+TUUmaFa4YD1Q+7bH9o5NCHQGPMqZCYJiNW6lIIS9z4= github.com/influxdata/usage-client v0.0.0-20160829180054-6d3895376368/go.mod h1:Wbbw6tYNvwa5dlB6304Sd+82Z3f7PmVZHVKU637d4po= github.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus= github.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc= github.com/jedisct1/go-minisign v0.0.0-20190909160543-45766022959e h1:UvSe12bq+Uj2hWd8aOlwPmoZ+CITRFrdit+sDGfAg8U= github.com/jedisct1/go-minisign v0.0.0-20190909160543-45766022959e/go.mod h1:G1CVv03EnqU1wYL2dFwXxW2An0az9JTl/ZsqXQeBlkU= -github.com/jessevdk/go-flags v0.0.0-20141203071132-1679536dcc89 h1:12K8AlpT0/6QUXSfV0yi4Q0jkbq8NDtIKFtF61AoqV0= github.com/jessevdk/go-flags v0.0.0-20141203071132-1679536dcc89/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= -github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg= github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo= -github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8= github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U= github.com/jmhodges/levigo v1.0.0 h1:q5EC36kV79HWeTBWsod3mG11EgStG3qArTKcvlksN1U= github.com/jmhodges/levigo v1.0.0/go.mod h1:Q6Qx+uH3RAqyK4rFQroq9RL7mdkABMcfhEI+nNuzMJQ= -github.com/jrick/logrotate v1.0.0 h1:lQ1bL/n9mBNeIXoTUoYRlK4dHuNJVofX9oWqBtPnSzI= github.com/jrick/logrotate v1.0.0/go.mod h1:LNinyqDIJnpAur+b8yyulnQw/wDuN1+BYKlTRt3OuAQ= -github.com/json-iterator/go v1.1.6 h1:MrUvLMLTMxbqFJ9kzlvat/rYZqZnW3u4wkLzWTaFwKs= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= -github.com/jstemmer/go-junit-report v0.9.1 h1:6QPYqodiu3GuPL+7mfx+NwDdp2eTkp9IfEUpgAwUN0o= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= -github.com/jsternberg/zap-logfmt v1.0.0 h1:0Dz2s/eturmdUS34GM82JwNEdQ9hPoJgqptcEKcbpzY= github.com/jsternberg/zap-logfmt v1.0.0/go.mod h1:uvPs/4X51zdkcm5jXl5SYoN+4RK21K8mysFmDaM/h+o= -github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/julienschmidt/httprouter v1.2.0 h1:TDTW5Yz1mjftljbcKqRcrYhd4XeOoI98t+9HbQbYf7g= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= -github.com/jung-kurt/gofpdf v1.0.3-0.20190309125859-24315acbbda5 h1:PJr+ZMXIecYc1Ey2zucXdR73SMBtgjPgwa31099IMv0= github.com/jung-kurt/gofpdf v1.0.3-0.20190309125859-24315acbbda5/go.mod h1:7Id9E/uU8ce6rXgefFLlgrJj/GYY22cpxn+r32jIOes= -github.com/jwilder/encoding v0.0.0-20170811194829-b4e1701a28ef h1:2jNeR4YUziVtswNP9sEFAI913cVrzH85T+8Q6LpYbT0= github.com/jwilder/encoding v0.0.0-20170811194829-b4e1701a28ef/go.mod h1:Ct9fl0F6iIOGgxJ5npU/IUOhOhqlVrGjyIZc8/MagT0= github.com/karalabe/usb v0.0.2 h1:M6QQBNxF+CQ8OFvxrT90BA0qBOXymndZnk5q235mFc4= github.com/karalabe/usb v0.0.2/go.mod h1:Od972xHfMJowv7NGVDiWVxk2zxnWgjLlJzE+F4F7AGU= -github.com/kisielk/errcheck v1.2.0 h1:reN85Pxc5larApoH1keMBiu2GWtPqXQ1nc9gx+jOU+E= -github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00= -github.com/kisielk/gotool v1.0.0 h1:AV2c/EiW3KqPNT9ZKl07ehoAGi4C5/01Cfbblndcapg= +github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23 h1:FOOIBWrEkLgmlgGfMuZT83xIwfPDxEI2OHu6xUmJMFE= github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4= -github.com/klauspost/compress v1.4.0 h1:8nsMz3tWa9SWWPL60G1V6CUsf4lLjWLTNEtibhe8gh8= github.com/klauspost/compress v1.4.0/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A= -github.com/klauspost/cpuid v0.0.0-20170728055534-ae7887de9fa5 h1:2U0HzY8BJ8hVwDKIzp7y4voR9CX/nvcfymLmg2UiOio= github.com/klauspost/cpuid v0.0.0-20170728055534-ae7887de9fa5/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek= -github.com/klauspost/crc32 v0.0.0-20161016154125-cb6bfca970f6 h1:KAZ1BW2TCmT6PRihDPpocIy1QTtsAsrx6TneU/4+CMg= github.com/klauspost/crc32 v0.0.0-20161016154125-cb6bfca970f6/go.mod h1:+ZoRqAPRLkC4NPOvfYeR5KNOrY6TD+/sAC3HXPZgDYg= -github.com/klauspost/pgzip v1.0.2-0.20170402124221-0bf5dcad4ada h1:3L+neHp83cTjegPdCiOxVOJtRIy7/8RldvMTsyPYH10= github.com/klauspost/pgzip v1.0.2-0.20170402124221-0bf5dcad4ada/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs= -github.com/konsorten/go-windows-terminal-sequences v1.0.1 h1:mweAR1A6xJ3oS2pRaGiHgQ4OO8tzTaLawm8vnODuwDk= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515 h1:T+h1c/A9Gawja4Y9mFVWj2vyii2bbUNDw3kt9VxK2EY= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= +github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= -github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI= -github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= -github.com/kr/pty v1.1.1 h1:VkoXIwSboBpnk99O/KFauAEILuNHv5DVFKZMBN/gUgw= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= -github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= -github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= -github.com/labstack/echo/v4 v4.2.1 h1:LF5Iq7t/jrtUuSutNuiEWtB5eiHfZ5gSe2pcu5exjQw= github.com/labstack/echo/v4 v4.2.1/go.mod h1:AA49e0DZ8kk5jTOOCKNuPR6oTnBS0dYiM4FW1e6jwpg= -github.com/labstack/gommon v0.3.0 h1:JEeO0bvc78PKdyHxloTKiF8BD5iGrH8T6MSeGvSgob0= github.com/labstack/gommon v0.3.0/go.mod h1:MULnywXg0yavhxWKc+lOruYdAhDwPK9wf0OL7NoOu+k= github.com/leanovate/gopter v0.2.9 h1:fQjYxZaynp97ozCzfOyOuAGOU4aU/z37zf/tOujFk7c= github.com/leanovate/gopter v0.2.9/go.mod h1:U2L/78B+KVFIx2VmW6onHJQzXtFb+p5y3y2Sh+Jxxv8= -github.com/lib/pq v1.0.0 h1:X5PMW56eZitiTeO7tKzZxFCSpbFZJtkMMooicw2us9A= github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= +github.com/magiconair/properties v1.8.6 h1:5ibWZ6iY0NctNGWo87LalDlEZ6R41TqbbDamhfG/Qzo= +github.com/magiconair/properties v1.8.6/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60= github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= -github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e h1:hB2xlXdHp/pmPZq0y3QnmWAArdw9PqbmotexnWx/FU8= github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= -github.com/matryer/moq v0.0.0-20190312154309-6cfb0558e1bd h1:HvFwW+cm9bCbZ/+vuGNq7CRWXql8c0y8nGeYpqmpvmk= github.com/matryer/moq v0.0.0-20190312154309-6cfb0558e1bd/go.mod h1:9ELz6aaclSIGnZBoaSLZ3NAl1VTufbOrXBPvtcy6WiQ= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= @@ -422,9 +319,7 @@ github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Ky github.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0= github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= -github.com/mattn/go-sqlite3 v1.11.0 h1:LDdKkqtYlom37fkvqs8rMPFKAMe8+SgjbwZ6ex1/A/Q= github.com/mattn/go-sqlite3 v1.11.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc= -github.com/mattn/go-tty v0.0.0-20180907095812-13ff1204f104 h1:d8RFOZ2IiFtFWBcKEHAFYJcPTf0wY5q0exFNJZVWa1U= github.com/mattn/go-tty v0.0.0-20180907095812-13ff1204f104/go.mod h1:XPvLUNfbS4fJH25nqRHfWLMa1ONC8Amw+mIA639KxkE= github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= @@ -432,13 +327,9 @@ github.com/mitchellh/mapstructure v1.4.1 h1:CpVNEelQCZBooIPDn+AR3NpivK/TIKU8bDxd github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= github.com/mitchellh/pointerstructure v1.2.0 h1:O+i9nHnXS3l/9Wu7r4NrEdwA2VFTicjUEN1uBnDo34A= github.com/mitchellh/pointerstructure v1.2.0/go.mod h1:BRAsLI5zgXmw97Lf6s25bs8ohIXc3tViBH44KcwB2g4= -github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= -github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI= github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/mschoch/smat v0.0.0-20160514031455-90eadee771ae h1:VeRdUYdCw49yizlSbMEn2SZ+gT+3IUKx8BqxyQdz+BY= github.com/mschoch/smat v0.0.0-20160514031455-90eadee771ae/go.mod h1:qAyveg+e4CE+eKJXWVjKXM4ck2QobLqTDytGJbLLhJg= -github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223 h1:F9x/1yl3T2AeKLr2AMdilSD8+f9bvMnNN8VS5iDtovc= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/naoina/go-stringutil v0.1.0 h1:rCUeRUHjBjGTSHl0VC00jUPLz8/F9dDzYI70Hzifhks= github.com/naoina/go-stringutil v0.1.0/go.mod h1:XJ2SJL9jCtBh+P9q5btrd/Ylo8XwT/h1USek5+NqSA0= @@ -446,7 +337,6 @@ github.com/naoina/toml v0.1.2-0.20170918210437-9fafd6967416 h1:shk/vn9oCoOTmwcou github.com/naoina/toml v0.1.2-0.20170918210437-9fafd6967416/go.mod h1:NBIhNtsFMo3G2szEBne+bO4gS192HuIYRqfvOWb4i1E= github.com/nxadm/tail v1.4.4 h1:DQuhQpB1tVlglWS2hLQ5OV6B5r8aGxSrPc5Qo6uTN78= github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= -github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4= github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec= github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY= @@ -463,24 +353,18 @@ github.com/opentracing/opentracing-go v1.0.2/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFSt github.com/opentracing/opentracing-go v1.0.3-0.20180606204148-bd9c31933947/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= github.com/opentracing/opentracing-go v1.1.0 h1:pWlfV3Bxv7k65HYwkikxat0+s3pV4bsqf19k25Ur8rU= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= -github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c h1:rp5dCmg/yLR3mgFuSOe4oEnDDmGLROTvMragMUXpTQw= -github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c/go.mod h1:X07ZCGwUbLaax7L0S3Tw4hpejzu63ZrrQiUe6W0hcy0= github.com/panjf2000/ants/v2 v2.4.5 h1:kcGvjXB7ea0MrzzszpnlVFthhYKoFxLi75nRbsq01HY= github.com/panjf2000/ants/v2 v2.4.5/go.mod h1:f6F0NZVFsGCp5A7QW/Zj/m92atWwOkY0OIhFxRNFr4A= -github.com/paulbellamy/ratecounter v0.2.0 h1:2L/RhJq+HA8gBQImDXtLPrDXK5qAj6ozWVK/zFXVJGs= github.com/paulbellamy/ratecounter v0.2.0/go.mod h1:Hfx1hDpSGoqxkVVpBi/IlYD7kChlfo5C6hzIHwPqfFE= github.com/peterh/liner v1.0.1-0.20180619022028-8c1271fcf47f/go.mod h1:xIteQHvHuaLYG9IFj6mSxM0fCKrs34IrEQUhOYuGPHc= github.com/peterh/liner v1.1.1-0.20190123174540-a2c9a5303de7 h1:oYW+YCJ1pachXTQmzR3rNLYGGz4g/UgFcjb28p/viDM= github.com/peterh/liner v1.1.1-0.20190123174540-a2c9a5303de7/go.mod h1:CRroGNssyjTd/qIG2FyxByd2S8JEAZXBl4qUrZf8GS0= -github.com/philhofer/fwd v1.0.0 h1:UbZqGr5Y38ApvM/V/jEljVxwocdweyH+vmYvRPBnbqQ= github.com/philhofer/fwd v1.0.0/go.mod h1:gk3iGcWd9+svBvR0sR+KPcfE+RNWozjowpeBVG3ZVNU= -github.com/pierrec/lz4 v2.0.5+incompatible h1:2xWsjqPFWcplujydGg4WmhC/6fZqK42wMM8aXeqhl0I= github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/term v0.0.0-20180730021639-bffc007b7fd5 h1:tFwafIEMf0B7NlcxV/zJ6leBIa81D3hgGSgsE5hCkOQ= github.com/pkg/term v0.0.0-20180730021639-bffc007b7fd5/go.mod h1:eCbImbZ95eXtAUIbLAuAVnBnwf83mjf6QIVH8SHYwqQ= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= @@ -502,43 +386,29 @@ github.com/prometheus/tsdb v0.7.1 h1:YZcsG11NqnK4czYLrWd9mpEuAJIHVQLwdrleYfszMAA github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= github.com/rcrowley/go-metrics v0.0.0-20190826022208-cac0b30c2563 h1:dY6ETXrvDG7Sa4vE8ZQG4yqWg6UnOcbqTAahkV813vQ= github.com/rcrowley/go-metrics v0.0.0-20190826022208-cac0b30c2563/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= -github.com/retailnext/hllpp v1.0.1-0.20180308014038-101a6d2f8b52 h1:RnWNS9Hlm8BIkjr6wx8li5abe0fr73jljLycdfemTp0= github.com/retailnext/hllpp v1.0.1-0.20180308014038-101a6d2f8b52/go.mod h1:RDpi1RftBQPUCDRw6SmxeaREsAaRKnOclghuzp/WRzc= github.com/rjeczalik/notify v0.9.1 h1:CLCKso/QK1snAlnhNR/CNvNiFU2saUtjV0bx3EwNeCE= github.com/rjeczalik/notify v0.9.1/go.mod h1:rKwnCoCGeuQnwBtTSPL9Dad03Vh2n40ePRrjvIXnJho= -github.com/rogpeppe/go-internal v1.3.0 h1:RR9dF3JtopPvtkroDZuVD7qquD0bnHlKSqaQhgwt8yk= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/rs/cors v1.7.0 h1:+88SsELBHx5r+hZ8TCkggzSstaWNbDvThkVK8H6f9ik= github.com/rs/cors v1.7.0/go.mod h1:gFx+x8UowdsKA9AchylcLynDq+nNFfI8FkUZdN/jGCU= -github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q= github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/segmentio/kafka-go v0.1.0/go.mod h1:X6itGqS9L4jDletMsxZ7Dz+JFWxM6JHfPOCvTvk+EJo= -github.com/segmentio/kafka-go v0.2.0 h1:HtCSf6B4gN/87yc5qTl7WsxPKQIIGXLPPM1bMCPOsoY= github.com/segmentio/kafka-go v0.2.0/go.mod h1:X6itGqS9L4jDletMsxZ7Dz+JFWxM6JHfPOCvTvk+EJo= -github.com/sergi/go-diff v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ= github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo= github.com/shirou/gopsutil v3.21.4-0.20210419000835-c7a38de76ee5+incompatible h1:Bn1aCHHRnjv4Bl16T8rcaFjYSrGrIZvpiGO6P3Q4GpU= github.com/shirou/gopsutil v3.21.4-0.20210419000835-c7a38de76ee5+incompatible/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA= -github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5IYyJwS/kOiWx8mHo= github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= -github.com/sirupsen/logrus v1.2.0 h1:juTguoYk5qI21pwyTXY3B3Y5cOTH3ZUyZCg1v/mihuo= github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= -github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d h1:zE9ykElWQ6/NYmHa3jpm/yHnI4xSofP+UP6SpjHcSeM= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= -github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= -github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72 h1:qLC7fQah7D6K1B0ujays3HV9gkFtllcxhzImRR7ArPQ= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= -github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8= github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= -github.com/spf13/cobra v0.0.3 h1:ZlrZ4XsMRm04Fr5pSFxBgfND2EBVa1nLpiy1stUsX/8= github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= -github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg= github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/status-im/keycard-go v0.0.0-20190316090335-8537d3370df4 h1:Gb2Tyox57NRNuZ2d3rmvB3pcmbu7O1RS3m8WRx7ilrg= github.com/status-im/keycard-go v0.0.0-20190316090335-8537d3370df4/go.mod h1:RZLeN1LMWmRsyYjvAu+I6Dm9QmlDaIIt+Y+4Kd7Tp+Q= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.2.0/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= @@ -555,7 +425,6 @@ github.com/tendermint/iavl v0.12.0 h1:xcaFAr+ycqCj7WN1RzL2EfcBioRDOHcU1oWcg83K02 github.com/tendermint/iavl v0.12.0/go.mod h1:EoKMMv++tDOL5qKKVnoIqtVPshRrEPeJ0WsgDOLAauM= github.com/tendermint/tendermint v0.31.11 h1:TIs//4WfEAG4TOZc2eUfJPI3T8KrywXQCCPnGAaM1Wo= github.com/tendermint/tendermint v0.31.11/go.mod h1:ymcPyWblXCplCPQjbOYbrF1fWnpslATMVqiGgWbZrlc= -github.com/tinylib/msgp v1.0.2 h1:DfdQrzQa7Yh2es9SuLkixqxuXS2SxsdYn0KbdrOGWD8= github.com/tinylib/msgp v1.0.2/go.mod h1:+d+yLhGm8mzTaHzB+wgMYrodPfmZrzkirds8fDWklFE= github.com/tklauser/go-sysconf v0.3.5 h1:uu3Xl4nkLzQfXNsWn15rPc/HQCJKObbt1dKJeWp3vU4= github.com/tklauser/go-sysconf v0.3.5/go.mod h1:MkWzOF4RMCshBAMXuhXJs64Rte09mITnppBXY/rYEFI= @@ -563,30 +432,21 @@ github.com/tklauser/numcpus v0.2.2 h1:oyhllyrScuYI6g+h/zUvNXNp1wy7x8qQy3t/piefld github.com/tklauser/numcpus v0.2.2/go.mod h1:x3qojaO3uyYt0i56EW/VUYs7uBvdl2fkfZFu0T9wgjM= github.com/tyler-smith/go-bip39 v1.0.1-0.20181017060643-dbb3b84ba2ef h1:wHSqTBrZW24CsNJDfeh9Ex6Pm0Rcpc7qrgKBiL44vF4= github.com/tyler-smith/go-bip39 v1.0.1-0.20181017060643-dbb3b84ba2ef/go.mod h1:sJ5fKU0s6JVwZjjcUEX2zFOnvq0ASQ2K9Zr6cf67kNs= -github.com/urfave/cli/v2 v2.3.0 h1:qph92Y649prgesehzOrQjdWyxFOp/QVM+6imKHad91M= github.com/urfave/cli/v2 v2.3.0/go.mod h1:LJmUH05zAU44vOAcrfzZQKsZbVcdbOG8rtL3/XcUArI= -github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw= github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc= github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8= -github.com/valyala/fasttemplate v1.2.1 h1:TVEnxayobAdVkhQfrfes2IzOB6o+z4roRkPF52WA1u4= github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ= -github.com/willf/bitset v1.1.3 h1:ekJIKh6+YbUIVt9DfNbkR5d6aFcFTLDRyJNAACURBg8= github.com/willf/bitset v1.1.3/go.mod h1:RjeCKbqT1RxIR/KWY6phxZiaY1IyutSBfGjNPySAYV4= -github.com/xlab/treeprint v0.0.0-20180616005107-d6fb6747feb6 h1:YdYsPAZ2pC6Tow/nPZOPQ96O3hm/ToAkGsPLzedXERk= github.com/xlab/treeprint v0.0.0-20180616005107-d6fb6747feb6/go.mod h1:ce1O1j6UtZfjr22oyGxGLbauSBp2YVXpARAosm7dHBg= -github.com/yuin/goldmark v1.2.1 h1:ruQGxdhGHe7FWOJPT0mKs5+pD2Xs1Bm/kdGlHO04FmM= +github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= go.etcd.io/bbolt v1.3.5 h1:XAzx9gjCb0Rxj7EoqcClPD1d5ZBxZJk0jbuoPHenBt0= go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= -go.opencensus.io v0.22.2 h1:75k/FF0Q2YM8QYo07VPddOLBslDt1MZOdEslOHvmzAs= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.uber.org/atomic v1.3.2 h1:2Oa65PReHzfn29GpvgsYwloV9AVFHPDk8tYxt2c2tr4= go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= -go.uber.org/multierr v1.1.0 h1:HoEmRHQPVSqub6w2z2d2EOVs2fjyFRGyofhKuyDq0QI= go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= -go.uber.org/zap v1.9.1 h1:XCJQEf3W6eZaVwhRBof6ImoYGJSITeKWsyeh3HFu/5o= go.uber.org/zap v1.9.1/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= golang.org/x/crypto v0.0.0-20170930174604-9419663f5a44/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= @@ -609,11 +469,9 @@ golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxT golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20191227195350-da58074b4299 h1:zQpM52jfKHG6II1ISZY1ZcpygvuSFZpLwfluuF89XOg= golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/image v0.0.0-20180708004352-c73c2afc3b81/go.mod h1:ux5Hcp/YLpHSI86hEcLt0YII63i6oz57MZXIpbrjZUs= golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= -golang.org/x/image v0.0.0-20190802002840-cff245a6509b h1:+qEpEAPhDZ1o0x3tHzZTQDArnOixOzGD9HUJfcg0mb4= golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= @@ -622,16 +480,14 @@ golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHl golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f h1:J5lckAjkw6qYlOZNj90mLYNTEKDvWeuc1yieZ8qUzUE= golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= -golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028 h1:4+4C/Iv2U4fMZBiMCc98MG1In4gJY5YRhtpDNeDeHWs= golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= +golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.4.2 h1:Gz96sIWK3OalVv/I/qNygP42zyoKp3xptRVCWRFEBvo= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -648,6 +504,7 @@ golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLL golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200813134508-3edf25e44fcc/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= @@ -661,7 +518,6 @@ golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAG golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw= golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -730,7 +586,6 @@ golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba h1:O8mE0/t419eoIwhTFpKVkHiT golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180525024113-a5b4c53f6e8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190206041539-40960b6deb8e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= @@ -754,7 +609,8 @@ golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtn golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200108203644-89082a384178/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.1.0 h1:po9/4sTYwZU9lPhi1tOrb4hCv3qrhiQ77LZfGa2OjwY= +golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -763,12 +619,9 @@ golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1N golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gonum.org/v1/gonum v0.0.0-20180816165407-929014505bf4/go.mod h1:Y+Yx5eoAFn32cQvJDxZx5Dpnq+c3wtXuadVZAcxbbBo= gonum.org/v1/gonum v0.0.0-20181121035319-3f7ecaa7e8ca/go.mod h1:Y+Yx5eoAFn32cQvJDxZx5Dpnq+c3wtXuadVZAcxbbBo= -gonum.org/v1/gonum v0.6.0 h1:DJy6UzXbahnGUf1ujUNkh/NEtK14qMo2nvlBPs4U5yw= gonum.org/v1/gonum v0.6.0/go.mod h1:9mxDZsDKxgMAuccQkewq682L+0eCu4dCN2yonUJTCLU= gonum.org/v1/netlib v0.0.0-20181029234149-ec6d1f5cefe6/go.mod h1:wa6Ws7BG/ESfp6dHfk7C6KdzKA7wR7u/rKwOGE66zvw= -gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0 h1:OE9mWmgKkjJyEmDAAtGMPjXu+YNeGvK9VTSHY6+Qihc= gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0/go.mod h1:wa6Ws7BG/ESfp6dHfk7C6KdzKA7wR7u/rKwOGE66zvw= -gonum.org/v1/plot v0.0.0-20190515093506-e2840ee46a6b h1:Qh4dB5D/WpoUUp3lSod7qgoyEHbDGPUWjIbnqdqqe1k= gonum.org/v1/plot v0.0.0-20190515093506-e2840ee46a6b/go.mod h1:Wt8AAjI+ypCyYX3nZBvf6cAIx93T+c/OS2HFAYskSZc= google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= @@ -776,13 +629,11 @@ google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEn google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.15.0 h1:yzlyyDW/J0w8yNFJIhiAJy4kq74S+1DOLdawELNxFMA= google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= -google.golang.org/appengine v1.6.5 h1:tycE03LOZYQNhDpS27tcQdAzLCVMaj7QT2SXxebnpCM= google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= @@ -812,15 +663,11 @@ google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miE google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= google.golang.org/protobuf v1.23.0 h1:4MY060fB1DLGMB/7MBTLnwQUY6+F09GEiz6SsrNqyzM= google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -gopkg.in/alecthomas/kingpin.v2 v2.2.6 h1:jMFz6MfLP0/4fUyZle81rXUoxOBFi19VUFKVDOQfozc= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= -gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= -gopkg.in/errgo.v2 v2.1.0 h1:0vLT13EuvQ0hNvakwLuFZ/jYrLp5F3kcWHXdRggjCE8= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= -gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4= gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce h1:+JknDZhAj8YMt7GC73Ei8pv4MzjDUNPHgQWJdtMAaDU= gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce/go.mod h1:5AcXVHNjg+BDxry382+8OKon8SEWiKktQR07RKPsv1c= @@ -836,21 +683,15 @@ gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU= gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= -gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo= -gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= -honnef.co/go/tools v0.1.3 h1:qTakTkI6ni6LFD5sBwwsdSO+AQqbSIxOauHTTQKZ/7o= honnef.co/go/tools v0.1.3/go.mod h1:NgwopIslSNH47DimFoV78dnkksY2EFtX0ajyb3K/las= -rsc.io/binaryregexp v0.2.0 h1:HfqmD5MEmC0zvwBuF187nq9mdnXjXsSivRiXN7SmRkE= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= -rsc.io/pdf v0.1.1 h1:k1MczvYDUvJBe93bYd7wrZLLUEcLZAuF824/I4e5Xr4= rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4= diff --git a/internal/ethapi/api.go b/internal/ethapi/api.go index 6473ec4535..58fa326344 100644 --- a/internal/ethapi/api.go +++ b/internal/ethapi/api.go @@ -1335,6 +1335,10 @@ func (s *PublicBlockChainAPI) GetDiffAccountsWithScope(ctx context.Context, bloc return result, err } +func (s *PublicBlockChainAPI) GetVerifyResult(ctx context.Context, blockNr rpc.BlockNumber, blockHash common.Hash, diffHash common.Hash) *core.VerifyResult { + return s.b.Chain().GetVerifyResult(uint64(blockNr), blockHash, diffHash) +} + // ExecutionResult groups all structured logs emitted by the EVM // while replaying a transaction in debug mode as well as transaction // execution status, the amount of gas used and the return value diff --git a/light/trie.go b/light/trie.go index 418ffaa3b6..d41536e069 100644 --- a/light/trie.go +++ b/light/trie.go @@ -50,6 +50,10 @@ type odrDatabase struct { backend OdrBackend } +func (db *odrDatabase) NoTries() bool { + return false +} + func (db *odrDatabase) OpenTrie(root common.Hash) (state.Trie, error) { return &odrTrie{db: db, id: db.id}, nil } @@ -190,6 +194,10 @@ func (t *odrTrie) do(key []byte, fn func() error) error { } } +func (db *odrTrie) NoTries() bool { + return false +} + type nodeIterator struct { trie.NodeIterator t *odrTrie diff --git a/node/node.go b/node/node.go index 0cb67b1ff5..d9bba741a7 100644 --- a/node/node.go +++ b/node/node.go @@ -589,12 +589,12 @@ func (n *Node) OpenDatabase(name string, cache, handles int, namespace string, r return db, err } -func (n *Node) OpenAndMergeDatabase(name string, cache, handles int, freezer, diff, namespace string, readonly, persistDiff bool) (ethdb.Database, error) { +func (n *Node) OpenAndMergeDatabase(name string, cache, handles int, freezer, diff, namespace string, readonly, persistDiff, pruneAncientData bool) (ethdb.Database, error) { chainDataHandles := handles if persistDiff { chainDataHandles = handles * chainDataHandlesPercentage / 100 } - chainDB, err := n.OpenDatabaseWithFreezer(name, cache, chainDataHandles, freezer, namespace, readonly, false, false) + chainDB, err := n.OpenDatabaseWithFreezer(name, cache, chainDataHandles, freezer, namespace, readonly, false, false, pruneAncientData) if err != nil { return nil, err } @@ -614,7 +614,7 @@ func (n *Node) OpenAndMergeDatabase(name string, cache, handles int, freezer, di // also attaching a chain freezer to it that moves ancient chain data from the // database to immutable append-only files. If the node is an ephemeral one, a // memory database is returned. -func (n *Node) OpenDatabaseWithFreezer(name string, cache, handles int, freezer, namespace string, readonly, disableFreeze, isLastOffset bool) (ethdb.Database, error) { +func (n *Node) OpenDatabaseWithFreezer(name string, cache, handles int, freezer, namespace string, readonly, disableFreeze, isLastOffset, pruneAncientData bool) (ethdb.Database, error) { n.lock.Lock() defer n.lock.Unlock() if n.state == closedState { @@ -633,7 +633,7 @@ func (n *Node) OpenDatabaseWithFreezer(name string, cache, handles int, freezer, case !filepath.IsAbs(freezer): freezer = n.ResolvePath(freezer) } - db, err = rawdb.NewLevelDBDatabaseWithFreezer(root, cache, handles, freezer, namespace, readonly, disableFreeze, isLastOffset) + db, err = rawdb.NewLevelDBDatabaseWithFreezer(root, cache, handles, freezer, namespace, readonly, disableFreeze, isLastOffset, pruneAncientData) } if err == nil { diff --git a/p2p/peer.go b/p2p/peer.go index 96783d83ac..144a32e7d8 100644 --- a/p2p/peer.go +++ b/p2p/peer.go @@ -235,6 +235,11 @@ func (p *Peer) Inbound() bool { return p.rw.is(inboundConn) } +// VerifyNode returns true if the peer is a verification connection +func (p *Peer) VerifyNode() bool { + return p.rw.is(verifyConn) +} + func newPeer(log log.Logger, conn *conn, protocols []Protocol) *Peer { protomap := matchProtocols(protocols, conn.caps, conn) p := &Peer{ @@ -345,11 +350,11 @@ func (p *Peer) handle(msg Msg) error { SendItems(p.rw, pongMsg) }) case msg.Code == discMsg: - var reason [1]DiscReason // This is the last message. We don't need to discard or // check errors because, the connection will be closed after it. - rlp.Decode(msg.Payload, &reason) - return reason[0] + var m struct{ R DiscReason } + rlp.Decode(msg.Payload, &m) + return m.R case msg.Code < baseProtocolLength: // ignore other base protocol messages return msg.Discard() diff --git a/p2p/peer_error.go b/p2p/peer_error.go index 393cc86b09..aad1a65c7a 100644 --- a/p2p/peer_error.go +++ b/p2p/peer_error.go @@ -54,7 +54,7 @@ func (pe *peerError) Error() string { var errProtocolReturned = errors.New("protocol returned") -type DiscReason uint +type DiscReason uint8 const ( DiscRequested DiscReason = iota diff --git a/p2p/server.go b/p2p/server.go index e076fd9348..c816e36762 100644 --- a/p2p/server.go +++ b/p2p/server.go @@ -114,6 +114,10 @@ type Config struct { // maintained and re-connected on disconnects. StaticNodes []*enode.Node + // Verify nodes are used as pre-configured connections which are always + // maintained and re-connected on disconnects. + VerifyNodes []*enode.Node + // Trusted nodes are used as pre-configured connections which are always // allowed to connect, even above the peer limit. TrustedNodes []*enode.Node @@ -218,6 +222,7 @@ const ( staticDialedConn inboundConn trustedConn + verifyConn ) // conn wraps a network connection with information gathered @@ -269,6 +274,9 @@ func (f connFlag) String() string { if f&inboundConn != 0 { s += "-inbound" } + if f&verifyConn != 0 { + s += "-verify" + } if s != "" { s = s[1:] } @@ -649,6 +657,9 @@ func (srv *Server) setupDialScheduler() { for _, n := range srv.StaticNodes { srv.dialsched.addStatic(n) } + for _, n := range srv.VerifyNodes { + srv.dialsched.addStatic(n) + } } func (srv *Server) maxInboundConns() int { @@ -934,6 +945,13 @@ func (srv *Server) checkInboundConn(remoteIP net.IP) error { // as a peer. It returns when the connection has been added as a peer // or the handshakes have failed. func (srv *Server) SetupConn(fd net.Conn, flags connFlag, dialDest *enode.Node) error { + // If dialDest is verify node, set verifyConn flags. + for _, n := range srv.VerifyNodes { + if dialDest == n { + flags |= verifyConn + } + } + c := &conn{fd: fd, flags: flags, cont: make(chan error)} if dialDest == nil { c.transport = srv.newTransport(fd, nil) diff --git a/params/config.go b/params/config.go index c8ebbf8f1d..2681da63df 100644 --- a/params/config.go +++ b/params/config.go @@ -294,9 +294,7 @@ var ( NielsBlock: big.NewInt(0), MirrorSyncBlock: big.NewInt(5184000), BrunoBlock: big.NewInt(13082000), - - // TODO modify blockNumber - EulerBlock: nil, + EulerBlock: big.NewInt(18907621), Parlia: &ParliaConfig{ Period: 3, @@ -342,9 +340,7 @@ var ( NielsBlock: big.NewInt(0), MirrorSyncBlock: big.NewInt(400), BrunoBlock: big.NewInt(400), - - // TODO - EulerBlock: nil, + EulerBlock: nil, Parlia: &ParliaConfig{ Period: 3, diff --git a/params/network_params.go b/params/network_params.go index 9311b5e2d5..cc322e86f4 100644 --- a/params/network_params.go +++ b/params/network_params.go @@ -53,6 +53,9 @@ const ( // CheckpointProcessConfirmations is the number before a checkpoint is generated CheckpointProcessConfirmations = 256 + // StableStateThreshold is the reserve number of block state save to disk before delete ancientdb + StableStateThreshold uint64 = 128 + // FullImmutabilityThreshold is the number of blocks after which a chain segment is // considered immutable (i.e. soft finality). It is used by the downloader as a // hard limit against deep ancestors, by the blockchain against deep reorgs, by diff --git a/params/version.go b/params/version.go index d4a04cfba2..0dc16e482a 100644 --- a/params/version.go +++ b/params/version.go @@ -23,7 +23,7 @@ import ( const ( VersionMajor = 1 // Major version component of the current release VersionMinor = 1 // Minor version component of the current release - VersionPatch = 10 // Patch version component of the current release + VersionPatch = 11 // Patch version component of the current release VersionMeta = "" // Version metadata to append to the version string ) diff --git a/tests/state_test_util.go b/tests/state_test_util.go index 3e66a1c55b..e71cd2bf3c 100644 --- a/tests/state_test_util.go +++ b/tests/state_test_util.go @@ -271,7 +271,7 @@ func MakePreState(db ethdb.Database, accounts core.GenesisAlloc, snapshotter boo var snaps *snapshot.Tree if snapshotter { - snaps, _ = snapshot.New(db, sdb.TrieDB(), 1, 128, root, false, true, false) + snaps, _ = snapshot.New(db, sdb.TrieDB(), 1, 128, root, false, true, false, false) } statedb, _ = state.New(root, sdb, snaps) return snaps, statedb diff --git a/trie/database.go b/trie/database.go index fcbade7f62..b8d1aa6eea 100644 --- a/trie/database.go +++ b/trie/database.go @@ -282,6 +282,7 @@ type Config struct { Cache int // Memory allowance (MB) to use for caching trie nodes in memory Journal string // Journal of clean cache to survive node restarts Preimages bool // Flag whether the preimage of trie key is recorded + NoTries bool } // NewDatabase creates a new trie database to store ephemeral trie content before diff --git a/trie/dummy_trie.go b/trie/dummy_trie.go new file mode 100644 index 0000000000..42e79a3719 --- /dev/null +++ b/trie/dummy_trie.go @@ -0,0 +1,101 @@ +// Copyright 2015 The go-ethereum Authors +// This file is part of the go-ethereum library. +// +// The go-ethereum library is free software: you can redistribute it and/or modify +// it under the terms of the GNU Lesser General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// The go-ethereum library is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Lesser General Public License for more details. +// +// You should have received a copy of the GNU Lesser General Public License +// along with the go-ethereum library. If not, see . + +package trie + +import ( + "fmt" + + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/ethdb" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/log" +) + +type EmptyTrie struct{} + +func (t *EmptyTrie) Prove(key []byte, fromLevel uint, proofDb ethdb.KeyValueWriter) error { + return nil +} + +// NewSecure creates a dummy trie +func NewEmptyTrie() *EmptyTrie { + return &EmptyTrie{} +} + +func (t *EmptyTrie) Get(key []byte) []byte { + return nil +} + +func (t *EmptyTrie) TryGet(key []byte) ([]byte, error) { + return nil, nil +} + +func (t *EmptyTrie) TryGetNode(path []byte) ([]byte, int, error) { + return nil, 0, nil +} +func (t *EmptyTrie) Update(key, value []byte) {} + +func (t *EmptyTrie) TryUpdate(key, value []byte) error { + return nil +} + +// Delete removes any existing value for key from the trie. +func (t *EmptyTrie) Delete(key []byte) { + if err := t.TryDelete(key); err != nil { + log.Error(fmt.Sprintf("Unhandled trie error: %v", err)) + } +} + +func (t *EmptyTrie) TryDelete(key []byte) error { + + return nil +} + +func (t *EmptyTrie) GetKey(shaKey []byte) []byte { + return nil +} + +func (t *EmptyTrie) Commit(onleaf LeafCallback) (root common.Hash, committed int, err error) { + return common.Hash{}, 0, nil +} + +func (t *EmptyTrie) Hash() common.Hash { + return common.Hash{} +} + +// Copy returns a copy of SecureTrie. +func (t *EmptyTrie) Copy() *EmptyTrie { + cpy := *t + return &cpy +} + +func (t *EmptyTrie) ResetCopy() *EmptyTrie { + cpy := *t + return &cpy +} + +// NodeIterator returns an iterator that returns nodes of the underlying trie. Iteration +// starts at the key after the given start key. +func (t *EmptyTrie) NodeIterator(start []byte) NodeIterator { + return nil +} + +// TryUpdateAccount abstract an account write in the trie. +func (t *EmptyTrie) TryUpdateAccount(key []byte, account *types.StateAccount) error { + return nil +}