WIP mega eof #29518

MariusVanDerWijden · 2024-04-12T10:52:17Z

Spec: https://github.com/ipsilon/eof/blob/main/spec/eof.md

holiman · 2024-04-17T10:58:25Z

cmd/evm/eofparser.go

@@ -161,3 +161,24 @@ func parseAndValidate(s string) (*vm.Container, error) {
 	}
 	return &c, nil
 }
+
+func eofDump(ctx *cli.Context) error {


we have cmd/rlpdump and cmd/abidump. How about a cmd/eofdump ?

Yep, will move

ah @fjl actually said today that he would prefer to have it a subcommand of evm

holiman · 2024-04-24T17:01:18Z

core/vm/validate.go

+			case op == DUPN:
+			case op == SWAPN:
+				arg := int(code[pos+1]) + 1
+				if want, have := arg, height; want >= have {


I find it unintuitive that want == have is an error.

holiman · 2024-08-26T11:07:45Z

Is this the latest and greatest eof branch/pr ?

core/vm/eips.go

holiman · 2024-09-04T13:55:08Z

core/vm/eips.go

+	if idx, overflow := idx.Uint64WithOverflow(); overflow || idx >= count {
+		// Index out-of-bounds, don't branch, just skip over immediate
+		// argument.
+		*pc += 1 + count*2


What is count is 0 -- won't this start executing on the immediate ? The spec says to pc += (max_index + 1) * 2, that is something like

*pc += 1 + (count+1)*2

???

I think I'm doing the right thing here, because we still have the +1 from the interpreter loop

core/vm/eips.go

holiman · 2024-09-06T08:11:39Z

core/vm/validate.go

+			backwards = worklist[idx].backwards
+		)
+		worklist = worklist[:idx]
+	outer:


Whenever we have these loops and labels and break outer, it's IMO usually better to branch off a method. So instead of

for .... outer: for .. switch break outer .... break outer bazonk()

we do

for .... inner() bazonk() func inner(){ for .. switch return //break outer .... return // break outer }

ah, this is old unused code. Please just nuke it, if we're not using it.

Co-authored-by: Mario Vega <[email protected]> Co-authored-by: lightclient <[email protected]>

Co-authored-by: lightclient <[email protected]> Co-authored-by: Mario Vega <[email protected]>

Fixes a few EOF validation issues in EEST and found by fuzzing. * subcontainers header with zero subcontainers is invalid * returning code sections require at least one RETF or qualified JUMPF * RETF stack must have stack enough for outputs * JUMPF must not jump into a section with more outputs * Check stack minimums on RJIMPI and RJUMPV

holiman · 2024-09-11T07:30:10Z

core/vm/jump_table.go

+		INVALID: {
+			execute:  opUndefined,
+			minStack: minStack(0, 0),
+			maxStack: maxStack(0, 0),
+			terminal: true,
+		},
 	}

 	// Fill all unassigned slots with opUndefined.
 	for i, entry := range tbl {
 		if entry == nil {
-			tbl[i] = &operation{execute: opUndefined, maxStack: maxStack(0, 0)}
+			tbl[i] = &operation{execute: opUndefined, maxStack: maxStack(0, 0), undefined: true}


The way this is defined, makes it so that a code-container containing nothing but the FE (invalid) opcode is a valid container.

$ go run ./cmd/eofdump/ eof --hex 0xef000101000402000100010400000000800000fe OK

That is, it behaves just like STOP:

[user@work go-ethereum]$ go run ./cmd/eofdump/ eof --hex 0xef00010100040200010001040000000080000000 OK

But different from an actual undefined opcode, like EF:

[user@work go-ethereum]$ go run ./cmd/eofdump/ eof --hex 0xef000101000402000100010400000000800000ef err(15): undefined instruction exit status 1

This seems wrong to me. @shemnon what do you say ?

It appears that this is covered by at least one of the common consensus-tests for eof. It seems very weird that we have one "blessed" undefined which acts differently from the actual undefined. Afaik, the idea with the designated undefined was that it will be left undefined, forever -- as opposed to non-designated undefineds, which may become used in the future.

The spec does mention this:https://github.com/ipsilon/eof/blob/main/spec/eof.md#stack-validation .

Terminating instructions:

ending function execution: RETF, JUMPF,

ending whole EVM execution: STOP, RETURN, RETURNCONTRACT, REVERT, INVALID.

I think this behaviour is utterly incomprehensible. Would love to know a bit more about the thoughts behind it. Why is it useful to allow INVALID?

IMO doing so violates the agreement/idea behind INVALID, by turning it into a unique opcode with a special purpose, as opposed to "one in the general soup of invalid/undefined operations".

The space that INVALID occupies is REVERT with penalties. We could easily replace it with PUSH0/PUSH0/REVERT but then all gas is not consumed. I think it is important that EOF have access to a "halt and catch fire" instruction.

Ok, thanks for the explanation. Re-reading what I wrote -- sorry if I sounded overly aggressive !

... 🍰

Fixes fill of opcode validiiy tests

* use corret stack heights for value and address in EXTCALL * New extCallGas function that handles min retained and min callee gas * Handle min gas failures via tempCallGas == 0 * remove stipend refund for EXT*CALL * Call Stack too deep should return 1

Handle Aux Data in EOF Create w/o validation failures

Allow Create Transactions

Some corner cases for create * Don't consume all gas with an invalid EOF contract * don't allow CREATE/CREATE2 opcodes to create EOF

Co-authored-by: Marius van der Wijden <[email protected]>

Eof/extcall fixes

@lightclient

The bulk of this PR is authored by @lightclient , in the original EOF-work. More recently, the code has been picked up and reworked for the new EOF specification, by @MariusVanDerWijden , in #29518, and also @shemnon has contributed with fixes. This PR is an attempt to start eating the elephant one small bite at a time, by selecting only the eof-validation as a standalone piece which can be merged without interfering too much in the core stuff. In this PR: - [x] Validation of eof containers, lifted from #29518, along with test-vectors from consensus-tests and fuzzing, to ensure that the move did not lose any functionality. - [x] Definition of eof opcodes, which is a prerequisite for validation - [x] Addition of `undefined` to a jumptable entry item. I'm not super-happy with this, but for the moment it seems the least invasive way to do it. A better way might be to go back and allowing nil-items or nil execute-functions to denote "undefined". - [x] benchmarks of eof validation speed --------- Co-authored-by: lightclient <[email protected]> Co-authored-by: Marius van der Wijden <[email protected]> Co-authored-by: Danno Ferrin <[email protected]>

@lightclient

The bulk of this PR is authored by @lightclient , in the original EOF-work. More recently, the code has been picked up and reworked for the new EOF specification, by @MariusVanDerWijden , in ethereum#29518, and also @shemnon has contributed with fixes. This PR is an attempt to start eating the elephant one small bite at a time, by selecting only the eof-validation as a standalone piece which can be merged without interfering too much in the core stuff. In this PR: - [x] Validation of eof containers, lifted from ethereum#29518, along with test-vectors from consensus-tests and fuzzing, to ensure that the move did not lose any functionality. - [x] Definition of eof opcodes, which is a prerequisite for validation - [x] Addition of `undefined` to a jumptable entry item. I'm not super-happy with this, but for the moment it seems the least invasive way to do it. A better way might be to go back and allowing nil-items or nil execute-functions to denote "undefined". - [x] benchmarks of eof validation speed --------- Co-authored-by: lightclient <[email protected]> Co-authored-by: Marius van der Wijden <[email protected]> Co-authored-by: Danno Ferrin <[email protected]>

@lightclient

The bulk of this PR is authored by @lightclient , in the original EOF-work. More recently, the code has been picked up and reworked for the new EOF specification, by @MariusVanDerWijden , in #29518, and also @shemnon has contributed with fixes. This PR is an attempt to start eating the elephant one small bite at a time, by selecting only the eof-validation as a standalone piece which can be merged without interfering too much in the core stuff. In this PR: - [x] Validation of eof containers, lifted from #29518, along with test-vectors from consensus-tests and fuzzing, to ensure that the move did not lose any functionality. - [x] Definition of eof opcodes, which is a prerequisite for validation - [x] Addition of `undefined` to a jumptable entry item. I'm not super-happy with this, but for the moment it seems the least invasive way to do it. A better way might be to go back and allowing nil-items or nil execute-functions to denote "undefined". - [x] benchmarks of eof validation speed --------- Co-authored-by: lightclient <[email protected]> Co-authored-by: Marius van der Wijden <[email protected]> Co-authored-by: Danno Ferrin <[email protected]>

@lightclient

The bulk of this PR is authored by @lightclient , in the original EOF-work. More recently, the code has been picked up and reworked for the new EOF specification, by @MariusVanDerWijden , in ethereum#29518, and also @shemnon has contributed with fixes. This PR is an attempt to start eating the elephant one small bite at a time, by selecting only the eof-validation as a standalone piece which can be merged without interfering too much in the core stuff. In this PR: - [x] Validation of eof containers, lifted from ethereum#29518, along with test-vectors from consensus-tests and fuzzing, to ensure that the move did not lose any functionality. - [x] Definition of eof opcodes, which is a prerequisite for validation - [x] Addition of `undefined` to a jumptable entry item. I'm not super-happy with this, but for the moment it seems the least invasive way to do it. A better way might be to go back and allowing nil-items or nil execute-functions to denote "undefined". - [x] benchmarks of eof validation speed --------- Co-authored-by: lightclient <[email protected]> Co-authored-by: Marius van der Wijden <[email protected]> Co-authored-by: Danno Ferrin <[email protected]>

@lightclient

The bulk of this PR is authored by @lightclient , in the original EOF-work. More recently, the code has been picked up and reworked for the new EOF specification, by @MariusVanDerWijden , in ethereum#29518, and also @shemnon has contributed with fixes. This PR is an attempt to start eating the elephant one small bite at a time, by selecting only the eof-validation as a standalone piece which can be merged without interfering too much in the core stuff. In this PR: - [x] Validation of eof containers, lifted from ethereum#29518, along with test-vectors from consensus-tests and fuzzing, to ensure that the move did not lose any functionality. - [x] Definition of eof opcodes, which is a prerequisite for validation - [x] Addition of `undefined` to a jumptable entry item. I'm not super-happy with this, but for the moment it seems the least invasive way to do it. A better way might be to go back and allowing nil-items or nil execute-functions to denote "undefined". - [x] benchmarks of eof validation speed --------- Co-authored-by: lightclient <[email protected]> Co-authored-by: Marius van der Wijden <[email protected]> Co-authored-by: Danno Ferrin <[email protected]>

@lightclient

The bulk of this PR is authored by @lightclient , in the original EOF-work. More recently, the code has been picked up and reworked for the new EOF specification, by @MariusVanDerWijden , in ethereum#29518, and also @shemnon has contributed with fixes. This PR is an attempt to start eating the elephant one small bite at a time, by selecting only the eof-validation as a standalone piece which can be merged without interfering too much in the core stuff. In this PR: - [x] Validation of eof containers, lifted from ethereum#29518, along with test-vectors from consensus-tests and fuzzing, to ensure that the move did not lose any functionality. - [x] Definition of eof opcodes, which is a prerequisite for validation - [x] Addition of `undefined` to a jumptable entry item. I'm not super-happy with this, but for the moment it seems the least invasive way to do it. A better way might be to go back and allowing nil-items or nil execute-functions to denote "undefined". - [x] benchmarks of eof validation speed --------- Co-authored-by: lightclient <[email protected]> Co-authored-by: Marius van der Wijden <[email protected]> Co-authored-by: Danno Ferrin <[email protected]>

lightclient added the prague label Apr 14, 2024

holiman reviewed Apr 17, 2024

View reviewed changes

holiman reviewed Apr 24, 2024

View reviewed changes

MariusVanDerWijden force-pushed the mega-eof branch 4 times, most recently from 801d48d to cdd4d1a Compare June 24, 2024 15:31

MariusVanDerWijden force-pushed the mega-eof branch 2 times, most recently from b516f42 to c8a1db2 Compare September 4, 2024 13:50

holiman reviewed Sep 4, 2024

View reviewed changes

holiman reviewed Sep 6, 2024

View reviewed changes

lightclient and others added 18 commits September 6, 2024 19:23

all: implement eip-7002 EL triggered withdrawal requests

cc3e859

all: implement eip-7251 consolidation requests

9d87ac7

Co-authored-by: Mario Vega <[email protected]> Co-authored-by: lightclient <[email protected]>

all: impl eip-7702

9db178c

Co-authored-by: lightclient <[email protected]> Co-authored-by: Mario Vega <[email protected]>

add delegation designator to witness

75a43ae

core: force add new delegation to accesslist

183e7b7

core/vm: add eof container

9354ef0

core/vm: add optional containerSections

1dccd90

core/types: implement mega-eof spec

07db5b8

core/vm: fixed bugs, made basic blockchain tests pass

d1317f2

core/vm: fix opExchange

b5af91a

core/vm: fix opSwapN

19e2965

core/vm: fix opRJumpV

fa6b1b9

core/vm: fix opReturnDataLoad

a1b359f

core/vm: fix opExtStaticCall

aa8e431

core/vm: only allow eof calls from opExtCall

e1b9f4a

core/vm: check for returnStack overflow in opCallf

3d8ca3c

core/vm: check for returnStack overflow in opCallf

b2ae978

core/vm: fix opExtCode{Size,Hash,Copy}

6c97819

holiman and others added 4 commits September 8, 2024 12:39

cmd/eofdump: simplify tests

71ffe6e

core/vm: move fix for JUMPF

cd7f772

core: fix rebase

6a6d049

MariusVanDerWijden force-pushed the mega-eof branch from 4be7308 to 6a6d049 Compare September 8, 2024 11:13

holiman added 2 commits September 8, 2024 13:15

cmd/eofdump: update tests

190d8b4

cmd/eofdump: more test vectors from fuzzing

cb57738

holiman reviewed Sep 11, 2024

View reviewed changes

holiman mentioned this pull request Sep 11, 2024

core/vm, cmd/evm: implement eof validation #30418

Merged

5 tasks

shemnon and others added 12 commits September 11, 2024 17:30

remove println on bad opcode

8d68a86

Fixes fill of opcode validiiy tests

EXTCALL test fixes

4a4d3b0

* use corret stack heights for value and address in EXTCALL * New extCallGas function that handles min retained and min callee gas * Handle min gas failures via tempCallGas == 0 * remove stipend refund for EXT*CALL * Call Stack too deep should return 1

EOFCREATE validation

389edbd

Handle Aux Data in EOF Create w/o validation failures

Create Transaction

4a93fe9

Allow Create Transactions

formatting

810397f

create transaction

c7efbf4

Some corner cases for create * Don't consume all gas with an invalid EOF contract * don't allow CREATE/CREATE2 opcodes to create EOF

cleanup from review comments

12bcb49

Fix dangling data in container for initcode

4392a1c

Update core/state_transition.go

995310c

Co-authored-by: Marius van der Wijden <[email protected]>

don't do beacon chain updates in state tests

740ad8c

Merge pull request #56 from shemnon/eof/extcall-fixes

2413e46

Eof/extcall fixes

cmd/eofdump: new testvectors from execution-spec-tests [email protected]

aee85e3

MariusVanDerWijden mentioned this pull request Sep 26, 2024

core/vm: implement eof opcodes #30511

Open

MariusVanDerWijden added osaka and removed prague labels Nov 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP mega eof #29518

WIP mega eof #29518

MariusVanDerWijden commented Apr 12, 2024 •

edited

Loading

holiman Apr 17, 2024

MariusVanDerWijden Apr 17, 2024

MariusVanDerWijden Sep 26, 2024

holiman Apr 24, 2024

holiman commented Aug 26, 2024

holiman Sep 4, 2024

MariusVanDerWijden Sep 4, 2024

holiman Sep 6, 2024

holiman Sep 6, 2024

holiman Sep 11, 2024

holiman Sep 11, 2024

holiman Sep 11, 2024 •

edited

Loading

shemnon Sep 11, 2024

holiman Sep 11, 2024

WIP mega eof #29518

Are you sure you want to change the base?

WIP mega eof #29518

Conversation

MariusVanDerWijden commented Apr 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

holiman commented Aug 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

holiman Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MariusVanDerWijden commented Apr 12, 2024 •

edited

Loading

holiman Sep 11, 2024 •

edited

Loading