fix(fw,tests): Fixes to EOF

[marioevz]

🗒️ Description

EOF classes use pydantic

EOF classes Section and Container now use pydantic BaseModel.

Opcodes with data portion are now subscriptable

Opcodes that have a data portion are now subscriptable:

• Op.PUSH1[1]
• Op.RJUMP[1]
• Op.RJUMPI[1](Op.ORIGIN)

Where the subscript is used to define the data portion of the opcode, and the call arguments can now be used exclusively for the stack.

Data portion can still be passed as the first argument to the call, so Op.PUSH1(1) == Op.PUSH1[1] == b"\x60\x01" but Op.PUSH1[1](1) will fail (we can begin to look forward to deprecate setting the data portion from the call arguments).

This is specially useful for Op.RJUMPV, where we can define each jump destination as a parameter of the subscript:

• Op.RJUMPV[5, 6, 7](Op.ORIGIN)

In this example, the RJUMPV is converted into bytecode with three possible jump destinations, +5, +6, or +7, and the max_index operand is automatically set to uint8(3), and Op.ORIGIN is executed to set the only required stack element.

Fixes to EOF tests

All tests usages of RJUMP, RJUMPI, RJUMPV and CALLF have been fixed to properly use subscripts.

Section.Code, Section.Data, Section.Container helpers

Three helpers have been added as class methods to the Section class, and can be used as helpers to create code, data or container sections respectively.

Section(kind=SectionKind.CODE, data=Op.STOP)

can now simply be:

Section.Code(code=Op.STOP)

Rest of the section arguments are passed along to the class creator.

Notes

Fixed Tox is still failing due to file tests/prague/eip3540_eof_v1/test_execution_function.py

🔗 Related Issues

✅ Checklist

• All: Set appropriate labels for the changes.
• All: Considered squashing commits to improve commit history.
• All: Added an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
• Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

[danceratopz]

tox -e tests-develop is failing as initial Prague support was added to t8n in
ethereum/go-ethereum@823719b, full support should be enabled with ethereum/go-ethereum#29557

(previously tests-develop was not actually filling any tests).

[winsvega]

validity_error=EOFException.DEFAULT_EXCEPTION,

logically this field does not belong to container. the container itself should be agnostic about its validity.
the validity lives on higher level

I generated tests from test_code_validation.py into json and verified it against evmone eof tool.
most invalid tests are ok, although I think the exception needs to be corrected on evmone side. to be more precise

the valid tests however are not generating at all, need to add more unit tests to pyspecs Container class and validate it with evmone. I posted TODO comments around the exceptions that I think need double checking

need more unit tests.
really not intuitive

custom_version=0x01,  # version can be overwritten for testing purposes (default is 0x01)
custom_terminator=bytes([0])

because I want to write
custom_terminator=0x01 or 0x00

but that does not work as it must be bytes.

[danceratopz]

Pydantic and Opcode changes LGTM! Pretty clever to hijack __getitem__ to create the RJUMPV look-up table.

[marioevz]

because I want to write custom_terminator=0x01 or 0x00

but that does not work as it must be bytes.

The reason we use bytes here is because you might want to add many zeros at the end for testing purposes and if you make this an integer you cannot specify more than one byte (0x0000 just resolves to 0 in python and the leading zeros are discarded). Maybe the correct name for this field should then be custom_terminator_bytes.

[winsvega]

because I want to write custom_terminator=0x01 or 0x00
but that does not work as it must be bytes.

The reason we use bytes here is because you might want to add many zeros at the end for testing purposes and if you make this an integer you cannot specify more than one byte (0x0000 just resolves to 0 in python and the leading zeros are discarded). Maybe the correct name for this field should then be custom_terminator_bytes.

then all the reset must be bytes too for consistancy.
its a pity 0x0001 is treated as integer in python