Add Ethereum compatibility test

[ulbqb]

Problem

Kaia isn't tested about Ethereum compatibility.
TBU

Solution

Kaia can be tested using test cases of Etheruem such as ethereum/tests and ethereum/execution-spec-test
TBU

Todo

• Add stateTest from ethereum/execution-spec-test for new EIPs
  • Shanghai: test: Add Shanghai tests by ethereum/execution-spec-tests #129
  • Cancun: test: Enable Shanghai, Cancun and Prague EIPs test by Cancun and Prague forks #154
  • Prague:
    • except 7702: test: Enable Shanghai, Cancun and Prague EIPs test by Cancun and Prague forks #154
    • 7702: feat: Introduce KIP-228 (EIP-7702) #172
• Investigating the difference between blockchainTest & stateTest: Add Ethereum compatibility test #152 (comment)
• Add blockchainTests from ethereum/execution-spec-test for new EIPs if it's needed
  • Shanghai
  • Cancun
  • Prague
• Add stateTest from ethereum/execution-spec-test for old EIPs
  • Frontier: tests: Enable state tests for all old EIPs by Cancun and Prague forks #164
  • Homestead: tests: Enable state tests for all old EIPs by Cancun and Prague forks #164
  • Byzantium: tests: Enable state tests for all old EIPs by Cancun and Prague forks #164
  • Constantinople: tests: Enable state tests for all old EIPs by Cancun and Prague forks #164
  • Istanbul: tests: Enable state tests for all old EIPs by Cancun and Prague forks #164
  • Berlin: tests: Enable state tests for all old EIPs by Cancun and Prague forks #164
• Add blockchainTest from ethereum/execution-spec-test for old EIPs if it's needed
  • Frontier
  • Homestead
  • Byzantium
  • Constantinople
  • Istanbul
  • Berlin
• Investigating the difference between ethereum/tests & ethereum/execution-spec-test
• Add tests using ethereum/tests if it's needed
• Add the detail to README

sub tasks:

• Replace execution-spec-test file with latest major version
• remove global variable, relaxPrecompileRangeForTest
• too many isTestExecutionSpecState branches, separate test runner like Run and runLegacy

future work:

• add similar tests to this for Kaia test

[ulbqb]

Investigating the difference between blockchainTest & stateTest

The Difference

StateTest is testing the post state after single transaction is executed. On the other hand, BlockchainTest is testing the post state after multiple transactions and multiple blocks are executed.

For details, let's take 7702 as an example. There are four blockchainTests for 7702.

• test_set_code_to_sstore_then_sload - Test the executing a simple SSTORE then SLOAD in two separate set-code transactions.
• test_set_code_to_system_contract - Test setting the code of an account to a pre-compile address.
• test_eoa_tx_after_set_code - Test sending a transaction from an EOA after code has been set to the account.
• test_reset_code - Test sending type-4 tx to reset the code of an account after code has been set to the account.

Basically, they all are the scenario where something is executed as another tx after code is set in the same block. In the StateTest, the state of first tx can be created using the pre state but this isn’t that the state is created during executing the same block. Considering this perspective, they are different and they are testing the continuous state transition in the same block.

BlockchainTest List

This shows what tests exist on BlockchainTest in the bottom of this page. Kaia is skipping eip4788_beacon_root and tests/cancun/eip4844_blobs, so they are removed. There are less tests than StateTests and tests to skip, but this contains tests to be executed such as 7702.

Appendix

BlockchainTest List

Click me

v3.0.0:

67 results - 21 files
src/cli/gentest.py:
  382      env: Environment,
  383:     blockchain_test: BlockchainTestFiller,
  384  ):
tests/cancun/eip6780_selfdestruct/test_dynamic_create2_selfdestruct_collision.py:
  245      call_create2_contract_at_the_end: bool,
  246:     blockchain_test: BlockchainTestFiller,
  247  ):
  489      recreate_on_first_tx: bool,
  490:     blockchain_test: BlockchainTestFiller,
  491  ):
tests/cancun/eip6780_selfdestruct/test_selfdestruct.py:
  630  def test_recreate_self_destructed_contract_different_txs(
  631:     blockchain_test: BlockchainTestFiller,
  632      env: Environment,
  906  def test_selfdestruct_created_same_block_different_tx(
  907:     blockchain_test: BlockchainTestFiller,
  908      eip_enabled: bool,
tests/cancun/eip7516_blobgasfee/test_blobgasfee_opcode.py:
  199  def test_blobbasefee_during_fork(
  200:     blockchain_test: BlockchainTestFiller,
  201      pre: Alloc,
tests/constantinople/eip1014_create2/test_recreate.py:
  21  def test_recreate(
  22:     blockchain_test: BlockchainTestFiller,
  23      pre: Alloc,
tests/frontier/opcodes/test_selfdestruct.py:
  13  @pytest.mark.valid_until("Homestead")
  14: def test_double_kill(blockchain_test: BlockchainTestFiller, pre: Alloc):
  15      """
tests/paris/security/test_selfdestruct_balance_bug.py:
  30  @pytest.mark.valid_from("Constantinople")
  31: def test_tx_selfdestruct_balance_bug(blockchain_test: BlockchainTestFiller, yul: YulCompiler):
  32      """
tests/prague/eip2935_historical_block_hashes_from_state/test_block_hashes.py:
   87  def test_block_hashes_history_at_transition(
   88:     blockchain_test: BlockchainTestFiller,
   89      pre: Alloc,
  206  def test_block_hashes_history(
  207:     blockchain_test: BlockchainTestFiller,
  208      pre: Alloc,
  312  def test_invalid_history_contract_calls(
  313:     blockchain_test: BlockchainTestFiller,
  314      pre: Alloc,
tests/prague/eip6110_deposits/test_deposits.py:
  697  def test_deposit(
  698:     blockchain_test: BlockchainTestFiller,
  699      pre: Alloc,
  953  def test_deposit_negative(
  954:     blockchain_test: BlockchainTestFiller,
  955      pre: Alloc,
tests/prague/eip7002_el_triggerable_withdrawals/test_withdrawal_requests.py:
  505  def test_withdrawal_requests(
  506:     blockchain_test: BlockchainTestFiller,
  507      blocks: List[Block],
  680      pre: Alloc,
  681:     blockchain_test: BlockchainTestFiller,
  682      requests: List[WithdrawalRequestInteractionBase],
tests/prague/eip7251_consolidations/test_consolidations.py:
  536  def test_consolidation_requests(
  537:     blockchain_test: BlockchainTestFiller,
  538      blocks: List[Block],
  733      pre: Alloc,
  734:     blockchain_test: BlockchainTestFiller,
  735      requests: List[ConsolidationRequestInteractionBase],
tests/prague/eip7685_general_purpose_el_requests/test_deposits_withdrawals_consolidations.py:
  220  def test_valid_deposit_withdrawal_consolidation_requests(
  221:     blockchain_test: BlockchainTestFiller,
  222      pre: Alloc,
  237  def test_valid_deposit_withdrawal_consolidation_request_from_same_tx(
  238:     blockchain_test: BlockchainTestFiller,
  239      pre: Alloc,
  363  def test_invalid_deposit_withdrawal_consolidation_requests(
  364:     blockchain_test: BlockchainTestFiller,
  365      pre: Alloc,
tests/prague/eip7702_eoa_code_tx/test_eoa_code_txs.py:
  116  def test_set_code_to_sstore_then_sload(
  117:     blockchain_test: BlockchainTestFiller,
  118      pre: Alloc,
tests/shanghai/eip4895_withdrawals/test_withdrawals.py:
  108          self,
  109:         blockchain_test: BlockchainTestFiller,
  110          post: dict,
  297          self,
  298:         blockchain_test: BlockchainTestFiller,
  299          test_case: str,
  365  
  366: def test_self_destructing_account(blockchain_test: BlockchainTestFiller, fork: Fork):
  367      """
  425  def test_newly_created_contract(
  426:     blockchain_test: BlockchainTestFiller,
  427      include_value_in_tx: bool,
  580  def test_zero_amount(
  581:     blockchain_test: BlockchainTestFiller,
  582      test_case: ZeroAmountTestCases,
  731  def test_withdrawing_to_precompiles(
  732:     blockchain_test: BlockchainTestFiller, precompile: int, amount: int, t8n: TransitionTool
  733  ):

[email protected]:

79 results - 25 files

src/cli/gentest.py:
  382      env: Environment,
  383:     blockchain_test: BlockchainTestFiller,
  384  ):

tests/cancun/eip1153_tstore/test_tstorage_clear_after_tx.py:
  31  def test_tstore_clear_after_deployment_tx(
  32:     blockchain_test: BlockchainTestFiller,
  33      pre: Alloc,

  79  def test_tstore_clear_after_tx(
  80:     blockchain_test: BlockchainTestFiller,
  81      pre: Alloc,

tests/cancun/eip6780_selfdestruct/test_selfdestruct.py:
  527  def test_recreate_self_destructed_contract_different_txs(
  528:     blockchain_test: BlockchainTestFiller,
  529      env: Environment,

  800  def test_selfdestruct_created_same_block_different_tx(
  801:     blockchain_test: BlockchainTestFiller,
  802      eip_enabled: bool,

tests/cancun/eip7516_blobgasfee/test_blobgasfee_opcode.py:
  204  def test_blobbasefee_during_fork(
  205:     blockchain_test: BlockchainTestFiller,
  206      pre: Alloc,

tests/constantinople/eip1014_create2/test_recreate.py:
  21  def test_recreate(
  22:     blockchain_test: BlockchainTestFiller,
  23      pre: Alloc,

tests/frontier/opcodes/test_selfdestruct.py:
  13  @pytest.mark.valid_until("Homestead")
  14: def test_double_kill(blockchain_test: BlockchainTestFiller, pre: Alloc):
  15      """

tests/paris/security/test_selfdestruct_balance_bug.py:
  29  @pytest.mark.valid_from("Constantinople")
  30: def test_tx_selfdestruct_balance_bug(blockchain_test: BlockchainTestFiller, pre: Alloc):
  31      """

tests/prague/eip2935_historical_block_hashes_from_state/test_block_hashes.py:
   87  def test_block_hashes_history_at_transition(
   88:     blockchain_test: BlockchainTestFiller,
   89      pre: Alloc,

  206  def test_block_hashes_history(
  207:     blockchain_test: BlockchainTestFiller,
  208      pre: Alloc,

  312  def test_invalid_history_contract_calls(
  313:     blockchain_test: BlockchainTestFiller,
  314      pre: Alloc,

tests/prague/eip6110_deposits/test_deposits.py:
  697  def test_deposit(
  698:     blockchain_test: BlockchainTestFiller,
  699      pre: Alloc,

  953  def test_deposit_negative(
  954:     blockchain_test: BlockchainTestFiller,
  955      pre: Alloc,

tests/prague/eip7002_el_triggerable_withdrawals/test_withdrawal_requests_during_fork.py:
  86  def test_withdrawal_requests_during_fork(
  87:     blockchain_test: BlockchainTestFiller,
  88      blocks: List[Block],

tests/prague/eip7002_el_triggerable_withdrawals/test_withdrawal_requests.py:
  508  def test_withdrawal_requests(
  509:     blockchain_test: BlockchainTestFiller,
  510      blocks: List[Block],

  684      fork: Fork,
  685:     blockchain_test: BlockchainTestFiller,
  686      requests: List[WithdrawalRequestInteractionBase],

tests/prague/eip7251_consolidations/test_consolidations_during_fork.py:
  86  def test_consolidation_requests_during_fork(
  87:     blockchain_test: BlockchainTestFiller,
  88      blocks: List[Block],

tests/prague/eip7251_consolidations/test_consolidations.py:
  537  def test_consolidation_requests(
  538:     blockchain_test: BlockchainTestFiller,
  539      blocks: List[Block],

  735      fork: Fork,
  736:     blockchain_test: BlockchainTestFiller,
  737      requests: List[ConsolidationRequestInteractionBase],

tests/prague/eip7685_general_purpose_el_requests/test_deposits_withdrawals_consolidations.py:
  223  def test_valid_deposit_withdrawal_consolidation_requests(
  224:     blockchain_test: BlockchainTestFiller,
  225      pre: Alloc,

  240  def test_valid_deposit_withdrawal_consolidation_request_from_same_tx(
  241:     blockchain_test: BlockchainTestFiller,
  242      pre: Alloc,

  469  def test_invalid_deposit_withdrawal_consolidation_requests(
  470:     blockchain_test: BlockchainTestFiller,
  471      pre: Alloc,

tests/prague/eip7685_general_purpose_el_requests/test_request_types.py:
  101  def test_invalid_request_type(
  102:     blockchain_test: BlockchainTestFiller,
  103      pre: Alloc,

tests/prague/eip7702_set_code_tx/test_set_code_txs.py:
   220  def test_set_code_to_sstore_then_sload(
   221:     blockchain_test: BlockchainTestFiller,
   222      pre: Alloc,

  2647  def test_set_code_to_system_contract(
  2648:     blockchain_test: BlockchainTestFiller,
  2649      pre: Alloc,

  2775  def test_eoa_tx_after_set_code(
  2776:     blockchain_test: BlockchainTestFiller,
  2777      pre: Alloc,

  2896  def test_reset_code(
  2897:     blockchain_test: BlockchainTestFiller,
  2898      pre: Alloc,

tests/shanghai/eip4895_withdrawals/test_withdrawals.py:
  124          pre: Alloc,
  125:         blockchain_test: BlockchainTestFiller,
  126          post: dict,

  135  def test_use_value_in_contract(
  136:     blockchain_test: BlockchainTestFiller,
  137      pre: Alloc,

  191  
  192: def test_balance_within_block(blockchain_test: BlockchainTestFiller, pre: Alloc):
  193      """

  301          self,
  302:         blockchain_test: BlockchainTestFiller,
  303          test_case: str,

  323  def test_many_withdrawals(
  324:     blockchain_test: BlockchainTestFiller,
  325      pre: Alloc,

  360  def test_self_destructing_account(
  361:     blockchain_test: BlockchainTestFiller,
  362      pre: Alloc,

  418  def test_newly_created_contract(
  419:     blockchain_test: BlockchainTestFiller,
  420      pre: Alloc,

  466  def test_no_evm_execution(
  467:     blockchain_test: BlockchainTestFiller,
  468      pre: Alloc,

  557  def test_zero_amount(
  558:     blockchain_test: BlockchainTestFiller,
  559      pre: Alloc,

  660  def test_large_amount(
  661:     blockchain_test: BlockchainTestFiller,
  662      pre: Alloc,

  701  def test_withdrawing_to_precompiles(
  702:     blockchain_test: BlockchainTestFiller,
  703      pre: Alloc,

[Mdaiki0730]

I will report the problem I encountered when running ETH blockchain testing on kaia.

Blockchain test flow

1. Calculate genesis from json
2. Compare the genesis block hash and state root with the expected in json
3. Create a Kaia blockchain object and insert a json block RLP into it.
4. After inserting the block, compare the expected state (post state) in the JSON with the actual state.
5. Validate the header of the inserted block

How to adopt to kaia

There are two solutions to this problem.
1. prepare following components

• A converter that converts blocks fields to Kaia blocks
  • A component that replicates the kaia state root
  • A component that calculates the Kaia header from the ETH header.
• A converter that converts Kaia blocks to eth blocks
  • A component that replicates the eth state root (can we use state_test?)
  • A component that calculates the eth header from the kaia header.
    This would require not only a converter before comparison, but also something like a mock chain to prepare the data to be inserted. Maintenance would also be costly.

2. compare only state

• Ignore the header and block only the txs to verify the state.
• In this pattern, steps 2 and 5 in the flow are skipped.
• There is an experiment in my fork repository.
  • Mdaiki0730@c6ae4c9

[Mdaiki0730]

Basically, they all are the scenario where something is executed as another tx after code is set in the same block. In the StateTest, the state of first tx can be created using the pre state but this isn’t that the state is created during executing the same block. Considering this perspective, they are different and they are testing the continuous state transition in the same block.

In my opinion, option 2 is the better option.
The reason is that it can achieve the purpose of blockchain testing presented by Shogo and requires only minor modifications.

Option 1 is expected to require a large amount of code to be added for the block converter. It may be necessary to use something like a mock chain to create the desired data using data obtained from JSON data.
Also, the purpose of checking what compatibility is being checked is unclear.

Could you give me your opinion? @kaiachain/core @shiki-tak @tnasu @ulbqb @da1suk8

[ulbqb]

@Mdaiki0730
I agree with you. The comparison of the post state is essential. I don't know whether or not other comparisons are needed now but I think you can start with option 2.