Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Upgrade] Go-Ethereum release v1.9.19 #1143

Closed
wants to merge 47 commits into from

Conversation

ricardolyn
Copy link
Contributor

TODO

Plan & Analyse

  • Review the Release Notes
  • Review PRs in the section below

As you review, list extra changes and/or tests to be implemented to ensure compatibility with GoQuorum specific features.

Build & Test

  • Pull and checkout PR branch locally, then merge GoQuorum master into this branch
  • Resolve conflicts, taking into account the prior analysis
  • Implement required changes until lint passes
  • Implement required changes until all unit tests pass
  • Implement required changes until acceptance tests pass
  • Implement extra changes and/or tests
  • Verify any left TODOs in the code

Extra Changes & Tests

  • Example title: example description of change/test

Go-Ethereum Release: Red Janey (v1.9.19)

  • Version: v1.9.19
  • Published: 2020-08-11T11:13:36Z

Release notes

Geth v1.9.19 is our regular biweekly maintenance release. Apart from the numerous bug-fixes however, it also ships a more deterministic transaction sort order during mining (FIFO). The goal is to reduce front-runner spam which abused miner randomness for transactions at the same price level.

  • Sort same-priced transactions by arrival time during mining to reduce front-running (#21358).
  • Persist the trie read cache on shutdown to speed up warmup time after a reboot (#20391).
  • Swap out the block fetcher of les/x to use the same mechanism as eth/6x (#20692).
  • Support setting custom HTTP headers on RCP clients e.g. for authentication (#21392).
  • Refactor node lifecycle management, merge GraphQL onto HTTP endpoint (#21105).
  • Print enode ID of a node too when doing an ENR dump via devp2p (#21270).
  • Trim the build paths to avoid leaking builder infos into the binaries (#21374).
  • Limit concurrent UPnP requests to avoid crashing certain routers (#21390).

Bugfixes:

  • Fix a potential data corruption by leaking out txpool internals into the miner (#21159).
  • Avoid a light server doing EVM calls during handshake while it's syncing (#21425).
  • Fix the trie clean cache size calculation if snapshots are disabled (#21416).
  • Revert an optimization regression in the JUMPDEST analysis (#21411).
  • Fix EIP 712 structured data signing corner cases (#21306, #21307).
  • Fix round-trip time calculation in the downloader (#21427, #21429).
  • Fix a panic in ethstats on the Görli stats page (#21404, #21434).
  • Fix a hang in the initial fast-sync's state sync phase (#21433).
  • Fix graceful LES server shutdown (#21426).

Note, if you were using GraphQL previously, it was moved to the HTTP RPC endpoint. The old --graphql.host and --graphql.port flags will not work any more. You might need to adjust your TOML config files accordingly too.

For a full rundown of the changes please consult the Geth 1.9.19 release milestone


As with all our previous releases, you can find the:

Codebase changes assessment

Legend

File Stats: (A) Added, (M) Modified and (R) Removed

Line Stats: (A) Added and (R) Removed

Assessment:

  • ✅ No conflict expected
  • ⚠ Review required to assess changes
  • ‼️ Conflicts expected and review required

35 Pull Requests

🔍 Link Title File Stats
M/A/R
Packages changed
(files changed)
Line Stats
A/R
Top 5 Changed Files
(lines changed)
#21379 Fix posible nil exeption 3/0/0
cmd/clef/docs (1)
cmd/faucet (1)
signer/storage (1)
5/5
signer/storage/aes_gcm_storage.go (12)
cmd/clef/docs/setup.md (4)
cmd/faucet/faucet.html (4)
⚠️ #20391 cmd, core, eth, light, trie: dump clean cache periodically 12/0/0
eth (4)
core (2)
cmd/geth (2)
cmd/utils (1)
core/state (1)
light (1)
trie (1)
128/24
trie/database.go (102)
eth/config.go (56)
core/blockchain.go (46)
cmd/utils/flags.go (32)
eth/gen_config.go (24)
#20692 les: implement new les fetcher 22/1/0
les (10)
eth/fetcher (3)
consensus/ethash (3)
les/utils (2)
consensus/clique (2)
eth (1)
cmd/geth (1)
consensus (1)
1325/1021
les/fetcher.go (2410)
eth/fetcher/block_fetcher_test.go (592)
les/fetcher_test.go (536)
eth/fetcher/block_fetcher.go (412)
les/utils/expiredvalue_test.go (154)
‼️ #21044 miner: refactor helper functions in worker.go 2/0/0
consensus/clique (1)
miner (1)
52/41
miner/worker.go (176)
consensus/clique/clique.go (10)
#21382 les: fix unittest 1/0/0
les (1)
4/4
les/fetcher_test.go (16)
⚠️ #21378 eth/downloader: remove eth62 3/0/0
eth/downloader (3)
5/60
eth/downloader/downloader_test.go (80)
eth/downloader/peer.go (42)
eth/downloader/downloader.go (8)
⚠️ #21374 build: use -trimpath flag when building executables 1/0/0
build (1)
4/2
build/ci.go (12)
#20878 core/bloombits: use single channel for shutdown 1/0/0
core/bloombits (1)
24/35
core/bloombits/matcher.go (118)
‼️ #21384 rawdb: fix comments to godoc convention format 1/0/0
core/rawdb (1)
2/2
core/rawdb/freezer.go (8)
⚠️ #21358 core: sort txs at the same gas price by received time 2/0/0
core/types (2)
86/22
core/types/transaction.go (110)
core/types/transaction_test.go (106)
#21391 rpc: remove silly use of ReadVarint in subscription ID generator 1/0/0
rpc (1)
6/3
rpc/subscription.go (18)
‼️ #21392 rpc: add SetHeader method to Client 3/0/0
rpc (3)
79/10
rpc/client_test.go (74)
rpc/http.go (74)
rpc/client.go (30)
#21306 TypedData.EncodeData should not reject a Domain without a ChainId 2/0/0
signer/core (2)
9/5
signer/core/signed_data_test.go (16)
signer/core/signed_data.go (12)
#21105 node: refactor package node 30/0/0
cmd/geth (6)
les (5)
cmd/utils (3)
eth (2)
graphql (2)
miner (2)
core/rawdb (1)
cmd/faucet (1)
cmd/wnode (1)
internal/ethapi (1)
mobile (1)
les/checkpointoracle (1)
cmd/p2psim (1)
console (1)
ethclient (1)
ethstats (1)
563/592
ethstats/ethstats.go (304)
cmd/utils/flags.go (228)
graphql/graphql_test.go (216)
eth/backend.go (188)
miner/stress_ethash.go (178)
#21307 signer: EIP 712, parse `bytes` and `bytesX` as hex strings + correct padding 2/0/0
signer/core (2)
118/3
signer/core/signed_data_internal_test.go (188)
signer/core/signed_data.go (54)
#21270 cmd/devp2p: print enode:// URL in enrdump 1/0/0
cmd/devp2p (1)
14/5
cmd/devp2p/enrcmd.go (38)
⚠️ #21159 core: avoid modification of accountSet cache in tx_pool 1/0/0
core (1)
5/2
core/tx_pool.go (14)
⚠️ #21404 ethstats: avoid concurrent write on websocket, fixes #21403 1/0/0
ethstats (1)
57/11
ethstats/ethstats.go (136)
#21390 p2p/nat: limit UPNP request concurrency 2/0/0
p2p/nat (2)
64/25
p2p/nat/natupnp.go (160)
p2p/nat/nat.go (18)
‼️ #21095 core: use errors.Is for consensus errors check 1/0/0
core (1)
6/6
core/blockchain.go (24)
⚠️ #21383 downloader: add eth65 tests 1/0/0
eth/downloader (1)
62/20
eth/downloader/downloader_test.go (164)
‼️ #21414 cmd/geth: fix import / export issues related to DB unavailability 1/0/0
cmd/geth (1)
4/2
cmd/geth/chaincmd.go (12)
#21411 core/vm: avoid map lookups for accessing jumpdest analysis 1/0/0
core/vm (1)
6/0
core/vm/contract.go (12)
‼️ #21415 cmd/geth: fixes db unavailability for chain commands 2/0/0
cmd/geth (2)
9/6
cmd/geth/chaincmd.go (26)
cmd/geth/config.go (4)
⚠️ #21419 core: Use maxQueuedTxAnns for to limit the number of transactions announced 1/0/0
eth (1)
1/1
eth/peer.go (4)
‼️ #21416 cmd/utils: grant snapshot cache to trie if disabled 2/0/0
cmd/utils (1)
eth (1)
3/2
eth/config.go (8)
cmd/utils/flags.go (2)
#21425 les: update checktime even if check fails 1/0/0
les/checkpointoracle (1)
5/3
les/checkpointoracle/oracle.go (16)
⚠️ #21427 eth/downloader: save the correct delivery time for state sync 1/0/0
eth/downloader (1)
10/9
eth/downloader/statesync.go (38)
⚠️ #21426 les: close all connected les-server when shutdown 3/0/0
les (3)
47/0
les/peer.go (78)
les/server.go (10)
les/server_handler.go (6)
⚠️ #21429 eth/downloader: set deliverytime on drops and timeouts too 1/0/0
eth/downloader (1)
3/0
eth/downloader/statesync.go (6)
⚠️ #21428 ethstats: overwrite old errors 1/0/0
ethstats (1)
2/3
ethstats/ethstats.go (10)
⚠️ #21433 eth/downloader: allow all timers to exit 1/0/0
eth/downloader (1)
1/7
eth/downloader/statesync.go (16)
⚠️ #21434 ethstats: split read and write lock, otherwise they'll lock up 1/0/0
ethstats (1)
18/14
ethstats/ethstats.go (64)
#21435 tests: add Berlin-definition identical to YOLOv1 1/0/0
tests (1)
14/0
tests/init.go (28)
⚠️ #21437 params: update CHTs for v1.9.19 1/0/0
params (1)
16/16
params/config.go (64)

120 Changed files

🔍 File Lines Changed Linked PR
les/fetcher.go 2410 #20692
‼️ node/node.go 1680
⚠️ node/node_test.go 1532
node/rpcstack.go 730
node/api_test.go 700
eth/fetcher/block_fetcher_test.go 592 #20692
p2p/testing/protocoltester.go 568
p2p/testing/protocolsession.go 566
les/fetcher_test.go 536 #21382
#20692
⚠️ ethstats/ethstats.go 442 #21404
#21105
#21428
#21434
⚠️ eth/fetcher/block_fetcher.go 412 #20692
‼️ node/api.go 358
‼️ cmd/utils/flags.go 262 #21105
#20391
#21416
node/utils_test.go 256
‼️ node/service.go 244
⚠️ eth/downloader/downloader_test.go 240 #21383
#21378
node/rpcstack_test.go 232
‼️ graphql/graphql_test.go 216 #21105
les/peer.go 206 #20692
#21426
node/service_test.go 196
signer/core/signed_data_internal_test.go 188 #21307
p2p/simulations/adapters/inproc.go 188
‼️ eth/backend.go 188 #21105
#20391
p2p/simulations/adapters/exec.go 180
miner/stress_ethash.go 178 #21105
‼️ miner/worker.go 176 #21044
miner/stress_clique.go 174 #21105
⚠️ whisper/whisperv6/whisper_test.go 172
graphql/service.go 170 #21105
p2p/nat/natupnp.go 160 #21390
les/utils/expiredvalue_test.go 154 #20692
p2p/testing/peerpool.go 134
core/bloombits/matcher.go 118 #20878
‼️ les/client.go 114 #21105
#20692
⚠️ les/server.go 112 #21105
#20692
#21426
whisper/whisperv6/whisper.go 112
⚠️ core/types/transaction.go 110 #21358
⚠️ core/types/transaction_test.go 106 #21358
⚠️ trie/database.go 102 #20391
‼️ cmd/geth/main.go 102 #21105
#20391
les/utils/expiredvalue.go 88 #20692
p2p/simulations/adapters/types.go 86
node/node_example_test.go 78
‼️ rpc/http.go 74 #21392
node/doc.go 74
⚠️ rpc/client_test.go 74 #21392
‼️ core/blockchain.go 70 #21095
#20391
signer/core/signed_data.go 66 #21307
#21306
⚠️ params/config.go 64 #21437
node/lifecycle.go 62
‼️ consensus/clique/clique.go 62 #21044
#20692
⚠️ consensus/consensus.go 60 #20692
⚠️ eth/downloader/statesync.go 60 #21427
#21429
#21433
⚠️ ethclient/ethclient_test.go 58 #21105
mobile/geth.go 58 #21105
les/client_handler.go 56 #20692
p2p/simulations/network_test.go 56
‼️ eth/config.go 56 #20391
#21416
⚠️ les/server_handler.go 54 #20692
#21426
les/commons.go 52 #21105
whisper/mailserver/server_test.go 50
⚠️ cmd/faucet/faucet.go 50 #21105
‼️ core/rawdb/freezer.go 44 #21384
#21105
‼️ eth/api_backend.go 44 #21105
les/api_test.go 42 #21105
whisper/whisperv6/filter_test.go 42
⚠️ eth/downloader/peer.go 42 #21378
‼️ cmd/geth/chaincmd.go 40 #21414
#21415
#21105
⚠️ node/config.go 40
‼️ consensus/ethash/consensus.go 40 #20692
cmd/devp2p/enrcmd.go 38 #21270
les/checkpointoracle/oracle.go 36 #21105
#21425
⚠️ cmd/geth/retesteth.go 36 #20692
p2p/simulations/examples/ping-pong.go 34
⚠️ cmd/geth/consolecmd.go 32 #21105
‼️ node/endpoints.go 30
⚠️ rpc/client.go 30 #21392
‼️ cmd/geth/config.go 28 #21415
#21105
tests/init.go 28 #21435
cmd/utils/flags_legacy.go 26 #21105
p2p/simulations/network.go 24
⚠️ eth/gen_config.go 24 #20391
node/errors.go 22
rpc/subscription.go 18 #21391
p2p/nat/nat.go 18 #21390
p2p/simulations/http_test.go 18
‼️ les/api_backend.go 18 #21105
signer/core/signed_data_test.go 16 #21306
console/console_test.go 16 #21105
⚠️ core/tx_pool.go 14 #21159
‼️ internal/ethapi/backend.go 14 #21105
⚠️ build/ci.go 12 #21374
core/vm/contract.go 12 #21411
‼️ core/state/database.go 12 #20391
‼️ cmd/geth/usage.go 12 #21105
#20391
signer/storage/aes_gcm_storage.go 12 #21379
⚠️ les/clientpool.go 12 #20692
cmd/wnode/main.go 10 #21105
⚠️ light/postprocess.go 8 #20391
⚠️ eth/downloader/downloader.go 8 #21378
⚠️ les/odr_test.go 8 #20692
‼️ eth/api_tracer.go 8 #20391
p2p/simulations/connect_test.go 8
⚠️ cmd/geth/dao_test.go 6 #21105
whisper/whisperv6/api_test.go 6
cmd/utils/cmd.go 4 #21105
⚠️ eth/peer.go 4 #21419
cmd/faucet/faucet.html 4 #21379
consensus/ethash/sealer.go 4 #20692
⚠️ core/genesis.go 4 #20391
⚠️ consensus/ethash/ethash.go 4 #20692
cmd/p2psim/main.go 4 #21105
⚠️ consensus/clique/api.go 4 #20692
p2p/simulations/test.go 4
cmd/clef/docs/setup.md 4 #21379
⚠️ params/version.go 4
⚠️ eth/handler.go 4 #20692
node/defaults.go 2
⚠️ les/test_helper.go 2 #20692
```` 0 #20692

karalabe and others added 30 commits July 27, 2020 14:58
* cmd, core, eth, light, trie: dump clean cache periodically

* eth: update config

* trie: minor fix

* core, trie: address comments

* eth: remove useless

* trie: print clean cache dump start too

Co-authored-by: Péter Szilágyi <[email protected]>
* cmd, consensus, eth, les: implement light fetcher

* les: address comment

* les: address comment

* les: address comments

* les: check td after delivery

* les: add linearExpiredValue for error counter

* les: fix import

* les: fix dead lock

* les: order announces by td

* les: encapsulate invalid counter

* les: address comment

* les: add more checks during the delivery

* les: fix log

* eth, les: fix lint

* eth/fetcher: address comment
This reduces complexity of some lengthy functions in worker.go,
making the code easier to read.
* init

notes

removed some mentions of eth62, bumped protocol err too old to >=63

* remove sanity checks and bump supported protocol version up to 63

* remove 62 tests, still need to add 65

* remove 65 tests
* Disable symbol table and DWARF generation by default.
Trimpath if compiling with Go >= 1.13

* Set Go to minimum version 1.13. Revert debug symbol changes.
This replaces the two-stage shutdown scheme with the one we
use almost everywhere else: a single quit channel signalling
termination.

Co-authored-by: Felix Lange <[email protected]>
core: sort txs at the same gas price by received time
Resolves #20163

Co-authored-by: Felix Lange <[email protected]>
…a ChainId (#21306)

* Do not check for a non-nil ChainId

* Add encoding test
This PR significantly changes the APIs for instantiating Ethereum nodes in
a Go program. The new APIs are not backwards-compatible, but we feel that
this is made up for by the much simpler way of registering services on
node.Node. You can find more information and rationale in the design
document: https://gist.github.com/renaynay/5bec2de19fde66f4d04c535fd24f0775.

There is also a new feature in Node's Go API: it is now possible to
register arbitrary handlers on the user-facing HTTP server. In geth, this
facility is used to enable GraphQL.

There is a single minor change relevant for geth users in this PR: The
GraphQL API is no longer available separately from the JSON-RPC HTTP
server. If you want GraphQL, you need to enable it using the
./geth --http --graphql flag combination.

The --graphql.port and --graphql.addr flags are no longer available.
…padding (#21307)

* Handle hex strings for bytesX types

* Add tests for parseBytes

* Improve tests

* Return nil bytes if error is non-nil

* Right-pad instead of left-pad bytes

* More tests
* core: avoid modification of accountSet cache in tx_pool

when runReorg, we may copy the dirtyAccounts' accountSet cache to promoteAddrs
in which accounts will be promoted, however, if we have reset request at the
same time, we may reuse promoteAddrs and modify the cache content which is
against the original intention of accountSet cache. So, we need to make a new
slice here to avoid modify accountSet cache.

* core: fix flatten condition + comment

Co-authored-by: Felix Lange <[email protected]>
This adds a lock around requests because some routers can't handle
concurrent requests. Requests are also rate-limited.
 
The Map function request a new mapping exactly when the map timeout
occurs instead of 5 minutes earlier. This should prevent duplicate mappings.
* eth65 tests

linted

* remove non-latest eth light tests
…1414)

* should fix import / export issues related to DB unavailability

* document reason for makeConfigNode

* fix comment

* comment consistency

* remove comments

* lint
core/vm: avoid map lookups for accessing jumpdest analysis
* chaincmd should make config nodes instead of full nodes

* add documentation for using makeConfigNode instead of makeFullNode;

* add documentation to functions

* code style
* cmd/utils: grant snapshot cache to trie if disabled

* eth: fix up default non-mainnet cache distribution
holiman and others added 17 commits August 7, 2020 10:57
les: update checktime even if check fails
eth/downloader: save the correct delivery time for state sync
* les: close all connected les-server when shutdown

* les: linter nitpick

Co-authored-by: Martin Holst Swende <[email protected]>
eth/downloader: set deliverytime on drops and timeouts too
eth/downloader: allow all timers to exit
ethstats: split read and write lock, otherwise they'll lock up
@CLAassistant
Copy link

CLAassistant commented Feb 24, 2021

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 4 committers have signed the CLA.

❌ timcooijmans
❌ hendrikhofstadt
❌ 6543
❌ robert-zaremba
You have signed the CLA already but the status is still pending? Let us recheck it.

@ricardolyn ricardolyn changed the base branch from master to upgrade/go-ethereum/v1.9.18-2021224170947 February 24, 2021 18:17
@ricardolyn ricardolyn self-assigned this Feb 25, 2021
Base automatically changed from upgrade/go-ethereum/v1.9.18-2021224170947 to master March 1, 2021 11:41
@ricardolyn
Copy link
Contributor Author

re-creating to generate right documentation

@ricardolyn ricardolyn closed this Mar 1, 2021
@ricardolyn ricardolyn deleted the upgrade/go-ethereum/v1.9.19-2021224181709 branch March 1, 2021 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.