Statediffing #7
Closed
Statediffing #7
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
statediff/service/service.go
Outdated
| parentHash := currentBlock.ParentHash() | ||
| parentBlock := sds.BlockChain.GetBlockByHash(parentHash) | ||
| if parentBlock == nil { | ||
| log.Error("Parent block is nil, skipping this block", |
There was a problem hiding this comment.
This error message is a bit misleading - we should reword it so that it is clear that the process will be stopped if parentBlock == nil, and not just skipping that single block.
i-norden
pushed a commit
that referenced
this pull request
Apr 15, 2019
* added tests for new abi encoding features (#4) * added tests from bytes32[][] and string[] * added offset to other types * formatting * Abi/dynamic types (#5) * Revert "Abi/dynamic types (#5)" (#6) This reverts commit dabca31. * Abi/dynamic types (#7) * some cleanup * Apply suggestions from code review apply suggestions Co-Authored-By: vedhavyas <[email protected]> * added better formatting (#8) * review chnages * better comments
i-norden
force-pushed
the
statediffing
branch
4 times, most recently
from
60e114f to
a055c2d
Compare
* port statediff from https://github.com/jpmorganchase/quorum/blob/9b7fd9af8082795eeeb6863d9746f12b82dd5078/statediff/statediff.go; minor fixes * integrating state diff extracting, building, and persisting into geth processes * work towards persisting created statediffs in ipfs; based off github.com/vulcanize/eth-block-extractor * Add a state diff service * Remove diff extractor from blockchain * Update imports * Move statediff on/off check to geth cmd config * Update starting state diff service * Add debugging logs for creating diff * Add statediff extractor and builder tests and small refactoring * Start to write statediff to a CSV * Restructure statediff directory * Pull CSV publishing methods into their own file * Reformatting due to go fmt * Add gomega to vendor dir * Remove testing focuses * Update statediff tests to use golang test pkg instead of ginkgo - builder_test - extractor_test - publisher_test * Use hexutil.Encode instead of deprecated common.ToHex * Remove OldValue from DiffBigInt and DiffUint64 fields * Update builder test * Remove old storage value from updated accounts * Remove old values from created/deleted accounts * Update publisher to account for only storing current account values * Update service loop and fetching previous block * Update testing - remove statediff ginkgo test suite file - move mocks to their own dir * Updates per go fmt * Updates to tests * Pass statediff mode and path in through cli * Return filename from publisher * Remove some duplication in builder * Remove code field from state diff output this is the contract byte code, and it can still be obtained by querying the db by the codeHash * Consolidate acct diff structs for updated & updated/deleted accts * Include block number in csv filename * Clean up error logging * Cleanup formatting, spelling, etc * Address PR comments * Add contract address and storage value to csv * Refactor accumulating account row in csv publisher * Add DiffStorage struct * Add storage key to csv * Address PR comments * Fix publisher to include rows for accounts that don't have store updates * Update builder test after merging in release/1.8 * Update test contract to include storage on contract intialization - so that we're able to test that storage diffing works for created and deleted accounts (not just updated accounts). * Factor out a common trie iterator method in builder
rmulhol
force-pushed
the
statediffing
branch
from
a055c2d to
7d75b70
Compare
* Open a trie from the in-memory database * Use a node's LeafKey as an identifier instead of the address It was proving difficult to find look the address up from a given path with a full node (sometimes the value wouldn't exist in the disk db). So, instead, for now we are using the node's LeafKey with is a Keccak256 hash of the address, so if we know the address we can figure out which LeafKey it matches up to. * Make sure that statediff has been processed before pruning * Use blockchain stateCache.OpenTrie for storage diffs * Clean up log lines and remove unnecessary fields from builder * Apply go fmt changes * Add a sleep to the blockchain test * Address PR comments * Address PR comments
…and paths (still need to make this optional)
…ing state diff payloads over an rpc websocket subscription
…r loop (may be missing something here)
… statediff processing is halted/paused until there is at least one subscriber listening for the results
rmulhol
force-pushed
the
statediffing
branch
from
7d75b70 to
ca79f6e
Compare
i-norden
pushed a commit
that referenced
this pull request
Aug 31, 2020
*** Changing Hex Trie to Binary Trie *** Note: This changes and/or comments out a bunch of tests, so if things break down the line, this is likely the cause!
i-norden
pushed a commit
that referenced
this pull request
Aug 31, 2020
* Get basic getStorage/setStorage stubs working * Clean up tests * Add state_manager * Add StateManager set & getStorage * Add state mananger create function * Add get & increment nonce * Add getCodeContractBytecode * Add GetCodeContractHash * Add getCodeContractHash to the state manager * Add associateCodeContract to state manager * Pass the tests * go fmt * Add stateTransition to test with * Fix tests * Test deploying contract with transition state * Call executeTransaction on contract deployment * Added ExecutionManager deployment * Get contract deployments working * Cleanup logging * Get stubbed ExecutionManager working * Get a simple contract to deploy through the ExecutionManager * Refactor simpleAbiEncode * Revert unnecessary changes * Remove comments * Revert changes outside of this PR * Revert changes outside of this PR * Revert changes outside of this PR * Fix broken tests * Move OVM bytecode & ABI into constants * Add crazy printlines * Remove crazy comments * Add a bunch of debug printlns * Add helper fn for applying msgs to the EVM * Update ExecutionManager bytecode * Shim CREATE for EM to use correct addr * Add SimpleStorage test * Add the EM/SM to all new states * Force all txs to be routed through the EM * Remove unused files * Remove unused comments * Increment nonce after failed tx * Add debug statements * Use evm.Time for timestamp * Change EM deployment, fix broken tests, clean up * Add an OVM test & remove printlns * Fix lint errors & remove final printlns * Final cleanup--remove some comments * Limiting Geth to one transaction per block (#3) * Limiting Geth to one transaction per block * Adding TransitionBatchBuilder to build & submit rollup blocks * Adding L1MessageSender to Transaction (#4) * Adding L1MessageSender to Transaction * Adding logic to omit L1MessageSender in encoding / decoding when nil and never use it in hash computation Co-authored-by: ben-chain <[email protected]> * Fixing Geth Tests (#6) Fixing broken tests, skipping tests we intentionally break, and configuring CI within Github Actions * Hex Trie -> Binary Trie (#7) *** Changing Hex Trie to Binary Trie *** Note: This changes and/or comments out a bunch of tests, so if things break down the line, this is likely the cause! * Ingest Block Batches (#8) Handling BlockBatches in Geth at `SendBlockBatches` endpoint (eth_sendBlockBatches) Other: * Adding PR template * Adding ability to set timestamp and making blocks use configured timestamp * Adding ability to encode original tx nonce in calldata * Adding L1MessageSender to Contract Creation Txs * Add L1MessageSender to Message * Increment nonce on CREATE failure * Fix bug where evm.Time=0 * Use state dump with hardcoded EM & SM addrs - ExecutionMgr address should always be 0x0000...dead0000 - StateMgr address should always be 0x0000...dead0001 * Move EM deployment into genesis block maker * Update EM contracts to latest version * Update EM to remove events * Fix the OVM tests * Skip an ungodly number of tests * Fix lint errors * Clean up logging * Cleanup more logs * Use local reference to state manager * Rename applyOvmToState(..) * Remove unneeded check * Clean up logging & add EM ABI panic * Add gas metering to SM & small refactor * Update core/vm/state_manager.go Co-authored-by: Kevin Ho <[email protected]> Co-authored-by: Mason Fischer <[email protected]> Co-authored-by: Will Meister <[email protected]> Co-authored-by: ben-chain <[email protected]> Co-authored-by: Kevin Ho <[email protected]>
i-norden
pushed a commit
that referenced
this pull request
Jun 7, 2021
…er (#7) (ethereum#22989) Co-authored-by: Gary Rong <[email protected]> Co-authored-by: Mikhail Kalinin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Opening this PR as a reference. This is the main branch for Vulcanize's statediffing for an archive node. It has been rebased with https://github.com/ethereum/go-ethereum/releases/tag/v1.8.21
Please note that this diff is very large because the master branch isn't yet up-to-date with geth v1.8.21. To see just state diffing additions, start at commit
52274e48a101b9e1c838b327a2b5c6087ad87857on Jan 28, 2019.