Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GETH 1.8.20 After update: Invalid Gas used #18358

Closed
jordimorris opened this issue Dec 23, 2018 · 10 comments
Closed

GETH 1.8.20 After update: Invalid Gas used #18358

jordimorris opened this issue Dec 23, 2018 · 10 comments

Comments

@jordimorris
Copy link

jordimorris commented Dec 23, 2018

Hi there,

please note that this is an issue tracker reserved for bug reports and feature requests.

For general questions please use the gitter channel or the Ethereum stack exchange at https://ethereum.stackexchange.com.

System information

Geth version: 1.8.20
OS & Version: Windows/Linux/OSX
Commit hash : (if develop)

Expected behaviour

synchronisation as usual

Actual behaviour

########## BAD BLOCK #########
Chain config: {ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7080000 Engine: ethash}

Number: 6936138
Hash: 0xd29f2839d79bc1fa46f59f05493fe252894b939bbc1aba8fa482cdc1729e996d


Error: nonce too high
##############################
 
WARN [12-23|19:37:59.082] Synchronisation failed, dropping peer    peer=ba1e40afac55b680 err="retrieved hash chain is invalid"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1c0 pc=0x46530e3]

goroutine 7047 [running]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).findAncestor(0xc0001ca1a0, 0xc051f820f0, 0xc03b659200, 0x0, 0x0, 0xc000042a70)
	/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:825 +0x15d3
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer(0xc0001ca1a0, 0xc051f820f0, 0xa2ce3b2da0887f27, 0x8c8598fba2677971, 0x6641fd958c42a2f9, 0x9b4e89349cb10222, 0xc067343e20, 0x0, 0x0)
	/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:437 +0x37d
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise(0xc0001ca1a0, 0xc03ecef920, 0x10, 0xa2ce3b2da0887f27, 0x8c8598fba2677971, 0x6641fd958c42a2f9, 0x9b4e89349cb10222, 0xc067343e20, 0x0, 0x0, ...)
	/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:406 +0x362
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).Synchronise(0xc0001ca1a0, 0xc03ecef920, 0x10, 0xa2ce3b2da0887f27, 0x8c8598fba2677971, 0x6641fd958c42a2f9, 0x9b4e89349cb10222, 0xc067343e20, 0x0, 0x4b189a0, ...)
	/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:324 +0x91
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).synchronise(0xc0148a41a0, 0xc057d8b980)
	/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/sync.go:200 +0x313
created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).handleMsg
	/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:682 +0xbed

@holiman
Copy link
Contributor

holiman commented Dec 30, 2018

There appears to be two problems here.
The first problem is that block 6936138/ 0xd29f2839d79bc1fa46f59f05493fe252894b939bbc1aba8fa482cdc1729e996d is rejected, despite being a canonical block. This indicates a database corruption.

The second problem is that the downloader panics, at https://github.com/ethereum/go-ethereum/blob/master/eth/downloader/downloader.go#L825 .

				header := d.lightchain.GetHeaderByHash(h) // Independent of sync mode, header surely exists
				if header.Number.Uint64() != check {
					p.log.Debug("Received non requested header", "number", header.Number, "hash", header.Hash(), "request", check)
					return 0, errBadPeer
}

So it appears that the comment is lying, and the hash is not available.

@holiman
Copy link
Contributor

holiman commented Dec 30, 2018

cc @zsfelfoldi , do you have any idea why the lightchain would fail to return the header at that point?

@holiman
Copy link
Contributor

holiman commented Dec 30, 2018

Although I guess if the database is corrupt, it's possible that reading the data from the database might fail. @jordimorris were there any other earlier error messages or uncontrolled shutdowns before this?

@jordimorris
Copy link
Author

no, I just updated as usual. I can run 1.8.19 and it syncs up perfectly. I switch to 1.8.20 and I got the errors again.

Don't you think it's an error from the latest version?

`INFO [01-03|10:29:33.677] IPC endpoint opened url=/Users/ethereum/Library/Ethereum/geth.ipc
INFO [01-03|10:29:33.679] HTTP endpoint opened url=http://127.0.0.1:8545 cors= vhosts=localhost
INFO [01-03|10:29:34.387] New local node record seq=34 id=686267e648ebe5da ip=62.57.56.92 udp=30303 tcp=30303
INFO [01-03|10:29:36.021] Mapped network port proto=udp extport=30303 intport=30303 interface="UPNP IGDv1-IP1"
INFO [01-03|10:29:36.032] Mapped network port proto=tcp extport=30303 intport=30303 interface="UPNP IGDv1-IP1"
INFO [01-03|10:29:53.671] Block synchronisation started
INFO [01-03|10:29:59.444] Importing heavy sidechain segment blocks=2048 start=6994837 end=6996884
INFO [01-03|10:30:26.499] Imported new chain segment blocks=2 txs=146 mgas=8.644 elapsed=27.053s mgasps=0.320 number=6994838 hash=ac1628…f295e0 age=1d8h7m cache=1.23mB
INFO [01-03|10:30:41.519] Imported new chain segment blocks=1 txs=102 mgas=7.995 elapsed=15.020s mgasps=0.532 number=6994839 hash=c57d15…6cd523 age=1d8h7m cache=2.12mB
INFO [01-03|10:30:52.457] Imported new chain segment blocks=1 txs=79 mgas=2.864 elapsed=10.937s mgasps=0.262 number=6994840 hash=8ca5a5…2f395c age=1d8h7m cache=2.62mB
INFO [01-03|10:31:02.142] Imported new chain segment blocks=1 txs=79 mgas=7.986 elapsed=9.684s mgasps=0.825 number=6994841 hash=8ced89…7b28e6 age=1d8h6m cache=3.42mB
INFO [01-03|10:31:14.638] Imported new chain segment blocks=3 txs=390 mgas=16.072 elapsed=12.495s mgasps=1.286 number=6994844 hash=0a06dd…550be6 age=1d8h5m cache=5.91mB
INFO [01-03|10:31:25.037] Imported new chain segment blocks=1 txs=88 mgas=7.992 elapsed=10.399s mgasps=0.769 number=6994845 hash=14fe4f…5afa06 age=1d8h5m cache=6.71mB
ERROR[01-03|10:31:32.550]
########## BAD BLOCK #########
Chain config: {ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7080000 Engine: ethash}

Number: 6994848
Hash: 0xf5d8c5004073394f49675048eee3461917dac13f3a471b1084670c1941e2d62d

Error: nonce too high
##############################

WARN [01-03|10:31:32.556] Synchronisation failed, dropping peer peer=1c0920442d7ffe5d err="retrieved hash chain is invalid"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1c0 pc=0x46530e3]

goroutine 1472 [running]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).findAncestor(0xc031424680, 0xc05920c960, 0xc03143d440, 0x0, 0x0, 0xc000044f70)
/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:825 +0x15d3
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer(0xc031424680, 0xc05920c960, 0xeb318d99be647ce7, 0xc3d4da0bc54d9ef2, 0x6d865571217d8340, 0xd61bb0e6aba1ef28, 0xc03efec4c0, 0x0, 0x0)
/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:437 +0x37d
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise(0xc031424680, 0xc058efee60, 0x10, 0xeb318d99be647ce7, 0xc3d4da0bc54d9ef2, 0x6d865571217d8340, 0xd61bb0e6aba1ef28, 0xc03efec4c0, 0x0, 0x0, ...)
/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:406 +0x362
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).Synchronise(0xc031424680, 0xc058efee60, 0x10, 0xeb318d99be647ce7, 0xc3d4da0bc54d9ef2, 0x6d865571217d8340, 0xd61bb0e6aba1ef28, 0xc03efec4c0, 0x0, 0xc02785d730, ...)
/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:324 +0x91
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).synchronise(0xc03a4215f0, 0xc0327e5500)
/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/sync.go:200 +0x313
created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).syncer
/private/tmp/ethereum-20181223-14360-e186tx/build/_workspace/src/github.com/ethereum/go-ethereum/eth/sync.go:155 +0x1df`

@simulanics
Copy link

simulanics commented Jan 6, 2019

Try adding a static-nodes file from the known seed-nodes list, and restart your server. The peer that your node is connecting to, has a corrupt database and apparently isn't connecting to other nodes to update it's own. Once your node rejects the block... Your node doesn't have any other peers to connect to either, to find the correct data. The nodes should be run with full sync mode to prevent "easy" corruption. Even using the static nodes file, your node will still seek out other trusted peers as well (if any exist..ie.. main net ethereum peer list). If you're running a "private" blockchain... You'll need at least 2 primary static seeds running to prevent future corruption of the"test/private" blockchain.

https://ethereum.stackexchange.com/questions/7261/how-to-use-static-nodes-json-trusted-nodes-json-to-prevent-connection-loss-on

@jordimorris
Copy link
Author

Simulanics, thank you for the time and detailed answer.
Version 19 works fine, up and running now. Why would I need to do anything to upgrade to version 20?

@simulanics
Copy link

Simulanics, thank you for the time and detailed answer.
Version 19 works fine, up and running now. Why would I need to do anything to upgrade to version 20?

Up until the latest release of Geth (Parity responded much faster to fix the issue), there was a bug with block validations for mining. Older clients did not check for the issue, and the updated clients do. Before, you could inject blocks with null data attached to a transaction, and since there was nothing to validate, they simply "slipped through" and were seen as valid transactions. The issue existed for a good 4-5 months before being fixed, unfortunately.

View these to learn more about the issue:

https://btcmanager.com/why-are-miners-mining-empty-blocks-and-what-affect-does-it-have-on-blockchain-networks/

https://ethereumworldnews.com/mining-empty-blocks-ethereum-could-be-a-problem/

Empty blocks are now dropped from validations before being added to the blockchain.

@cblk
Copy link

cblk commented Feb 22, 2019

I have the same issue, just try upgrading Geth to 1.8.21 or higher
https://blog.gophersland.com/upgrading-geth-to-constantinople-1-8-20-hardfork

@jordimorris
Copy link
Author

updated geth to the last version, 1.8.23 and it sorted out all the errors.

Thanks!

@holiman
Copy link
Contributor

holiman commented Jun 19, 2019

I think the root cause was a corrupt db, closing this

@holiman holiman closed this as completed Jun 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants