evmtool crashes on unexpected stateroot

[namiloh]

Steps to Reproduce (Bug)

Run a test with invalid expected root, see that the trace does not contain the stateroot at the end

martin@worknuk:~/workspace/goevmlab/evms/testdata$ /home/martin/workspace/besu-vm --json --nomemory --notime state-test ./cases/00000936-mixed-1.json 2>/dev/null
...omitted for brevity...
{"pc":82,"op":8,"gas":"0x79bc20","gasCost":"0x8","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d","0xb94f5374fce5edbc8e2a8697c15331677e6ebf0b","0x10"],"depth":1,"refund":0,"opName":"ADDMOD"}
{"pc":83,"op":62,"gas":"0x79bc18","gasCost":"0x0","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0xb94f5374fce5edbc8e2a8697c15331677e6ebf1b"],"depth":1,"refund":0,"opName":"RETURNDATACOPY","error":"Out of bounds"}

json test file can be found here: https://github.com/holiman/goevmlab/tree/master/evms/testdata/cases, and the expected stdout should end with a stateroot, as can be seen here

The reason why the stateroot is not shown can be seen if we look at the stderr:

martin@worknuk:~/workspace/goevmlab/evms/testdata$ /home/martin/workspace/besu-vm --json --nomemory --notime state-test ./cases/00000936-mixed-1.json 1>/dev/null
java.lang.RuntimeException: World State Root does not match expected value, header 0x0000000000000000000000000000000000000000000000000000000000000000 calculated 0xd14c10ed22a1cfb642e374be985ac581c39f3969bd59249e0405aca3beb47a47
	at org.hyperledger.besu.ethereum.bonsai.worldview.BonsaiWorldState.persist(BonsaiWorldState.java:398)
	at org.hyperledger.besu.evmtool.StateTestSubCommand.traceTestSpecs(StateTestSubCommand.java:266)
	at org.hyperledger.besu.evmtool.StateTestSubCommand.lambda$executeStateTest$0(StateTestSubCommand.java:174)
	at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
	at org.hyperledger.besu.evmtool.StateTestSubCommand.executeStateTest(StateTestSubCommand.java:174)
	at org.hyperledger.besu.evmtool.StateTestSubCommand.run(StateTestSubCommand.java:157)
	at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
	at picocli.CommandLine.access$1500(CommandLine.java:148)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
	at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
	at picocli.CommandLine.execute(CommandLine.java:2170)
	at org.hyperledger.besu.evmtool.EvmToolCommand.execute(EvmToolCommand.java:271)
	at org.hyperledger.besu.evmtool.EvmToolCommand.execute(EvmToolCommand.java:237)
	at org.hyperledger.besu.evmtool.EvmTool.main(EvmTool.java:23)

It crashes, which makes it not compatible with the fuzzing-framework.

Commit version 4f49ec9

[garyschulte]

can you retest with main since the merge of #5960 🙏

[namiloh]

at f0b2b56

martin@worknuk:~/workspace$ ./besu/ethereum/evmtool/build/install/evmtool/bin/evm --version
Hyperledger Besu evm 23.10.1-dev-f0b2b560
martin@worknuk:~/workspace$ ./besu/ethereum/evmtool/build/install/evmtool/bin/evm --nomemory --notime --json state-test /tmp/00000000-mixed-0.json
{"pc":0,"op":96,"gas":"0x79bef8","gasCost":"0x3","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":2,"op":96,"gas":"0x79bef5","gasCost":"0x3","memSize":0,"stack":["0x6f"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":4,"op":96,"gas":"0x79bef2","gasCost":"0x3","memSize":0,"stack":["0x6f","0xe6"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":6,"op":96,"gas":"0x79beef","gasCost":"0x3","memSize":0,"stack":["0x6f","0xe6","0xfd"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":8,"op":96,"gas":"0x79beec","gasCost":"0x3","memSize":0,"stack":["0x6f","0xe6","0xfd","0x56"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":10,"op":96,"gas":"0x79bee9","gasCost":"0x3","memSize":0,"stack":["0x6f","0xe6","0xfd","0x56","0xb0"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":12,"op":96,"gas":"0x79bee6","gasCost":"0x3","memSize":0,"stack":["0x6f","0xe6","0xfd","0x56","0xb0","0x25"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":14,"op":91,"gas":"0x79bee3","gasCost":"0x1","memSize":0,"stack":["0x6f","0xe6","0xfd","0x56","0xb0","0x25","0x4f"],"depth":1,"refund":0,"opName":"JUMPDEST"}
{"pc":15,"op":129,"gas":"0x79bee2","gasCost":"0x3","memSize":0,"stack":["0x6f","0xe6","0xfd","0x56","0xb0","0x25","0x4f"],"depth":1,"refund":0,"opName":"DUP2"}
{"pc":16,"op":143,"gas":"0x79bedf","gasCost":"0x0","memSize":0,"stack":["0x6f","0xe6","0xfd","0x56","0xb0","0x25","0x4f","0x25"],"depth":1,"refund":0,"opName":"DUP16","error":"Stack underflow"}
java.lang.RuntimeException: World State Root does not match expected value, header 0x0000000000000000000000000000000000000000000000000000000000000000 calculated 0x15aac4e32c9fdfd09b6b1623fd19be9e4b59f636765db19bf31d444713ce57c3
	at org.hyperledger.besu.ethereum.bonsai.worldview.BonsaiWorldState.verifyWorldStateRoot(BonsaiWorldState.java:435)
	at org.hyperledger.besu.ethereum.referencetests.BonsaiReferenceTestWorldState.verifyWorldStateRoot(BonsaiReferenceTestWorldState.java:93)
	at org.hyperledger.besu.ethereum.bonsai.worldview.BonsaiWorldState.persist(BonsaiWorldState.java:393)
	at org.hyperledger.besu.evmtool.StateTestSubCommand.traceTestSpecs(StateTestSubCommand.java:266)
	at org.hyperledger.besu.evmtool.StateTestSubCommand.lambda$executeStateTest$0(StateTestSubCommand.java:174)
	at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
	at org.hyperledger.besu.evmtool.StateTestSubCommand.executeStateTest(StateTestSubCommand.java:174)
	at org.hyperledger.besu.evmtool.StateTestSubCommand.run(StateTestSubCommand.java:157)
	at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
	at picocli.CommandLine.access$1500(CommandLine.java:148)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
	at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
	at picocli.CommandLine.execute(CommandLine.java:2170)
	at org.hyperledger.besu.evmtool.EvmToolCommand.execute(EvmToolCommand.java:277)
	at org.hyperledger.besu.evmtool.EvmToolCommand.execute(EvmToolCommand.java:243)
	at org.hyperledger.besu.evmtool.EvmTool.main(EvmTool.java:23)

[namiloh]

the test

$ cat /tmp/00000000-mixed-0.json
{"00000000-mixed-0":{"env":{"currentCoinbase":"b94f5374fce5edbc8e2a8697c15331677e6ebf0b","currentDifficulty":"0x200000","currentRandom":"0x0000000000000000000000000000000000000000000000000000000000020000","currentGasLimit":"0x26e1f476fe1e22","currentNumber":"0x1","currentTimestamp":"0x3e8","previousHash":"0x044852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d","currentBaseFee":"0x10"},"pre":{"0x00000000000000000000000000000000000000f1":{"code":"0x606f60e660fd605660b06025604f5b818f7e6a631d09457df0180834090815301979013c8a3a18671118117d0052648e6190909b0a8a397c15616133363b198040f48dfa320661608671744604f37860413d83431a3f883a5f937316fa43388408418050331a82153532513d462073570b156ff4345a430a6cfd42400a04336a8e489a8a5f537755531d1b05a2406a3f8d9a1d35a4043c881570a009368684388262338c886365fd606f681d916b6c6006583709467d8da44573013b548f868d09708a676b58643a7c4806835f443b1d01790578fe463ba23b627e5731477998189319718957771962568532777ba185363739841c6600609d02f567591338137f2042760bf43568635353556a68866002313317566a3ef27a090214576d7d82409d8e7c807f546c3e386d8b88a458091a12146c3a668700f0171a168e5a68421b711b037671345539666658537017038c51fa75360420816c9f727c7d3d0194471b389a5f065ff26e31a29633889982813115853852ff067e74847730f57a48877f3743116d5030369f83021a20004146f20308973e5f769486321a51a3f1395b977b126e67816e1103fa7317a17635716b3c6d60476f66062061076c783b97f192066799891d136741357d8c1b680058896a760a1646056a93133e5b350a3e3f480914527b6f1d4754a30699695061313ba39d8587169e8714917d19641a6144635f20166e613d529036000969ff3c1672145f5666328760148104387c13636c0a3e091a148d79f2566842386641475609ff777f3c3f0a358e68130874088478f511018518324654323a166f9d7c1574313413a16a8a7af207fe96813f740b0b484715544747687814a29d65197e6f518b6582603788334034186178646a1d916565084270586e99f1f13b3b5182081216369e52547a751ca07335604076316a6e0648054802037820f05ba0206a013e8553a16266558c6b0730423b333068517b7e617a5580807b3a548e6d316491538a16546c753b009d4038657c68763cfe7f6c770b4271140704371c8e95487cf5847435897805069399140b041c693b79ff6c038718393a955b781d6e8f8574553c02129f57063b8d471c16461143677e306d6b42984196476b9b5214420b6b7a6b55306e2085987a50688c748c8c798165140b6b168e477f775103468b69f0730556398f158a8a6a86527a383bf265360b7f88543d13821a6ef48a19997f5f90f59802422047310115154213017a8b575941611d750a8543106507147c5087f47f68958e0b4009175b18063640734444581550658b317018478a98455a3983839973a08d813b1a847108663d68a16b867a1b61728235626ff3834446144316558c685001f3751103874300931a54f1330852567772704787993f5a691c3e7218868172459e3c8a8d67373f3b778f52","storage":{"0x0000000000000000000000000000000000000000000000000000000000000000":"0x0000000000000000000000000000000000000000000000000000000000000012","0x0000000000000000000000000000000000000000000000000000000000000001":"0x0000000000000000000000000000000000000000000000000000000000000004"},"balance":"0x0","nonce":"0x0"},"0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b":{"code":"0x","storage":{},"balance":"0xffffffffff","nonce":"0x0"}},"transaction":{"gasPrice":"0x10","nonce":"0x0","to":"0x00000000000000000000000000000000000000f1","data":["0x21bcb59ed4649df8f5ac834ae8fa0eb1"],"gasLimit":["0x7a1200"],"value":["0x1716"],"sender":"0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b","secretKey":"0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"},"out":"0x","post":{"Shanghai":[{"hash":"0x0000000000000000000000000000000000000000000000000000000000000000","logs":"0x0000000000000000000000000000000000000000000000000000000000000000","indexes":{"data":0,"gas":0,"value":0}}]}}}

[holiman]

Afaict this PR broke it again: #6013

[namiloh]

Seems fixed