Progress tracking

[cassc]

This list will be updated while fixing the tests and benchmarks

• deterministic_deploy: REVM has removed CREATE2 support from the transaction. The recommended alternative solution is to "change address after execution". This will introduce a breaking change in the Python API.

(from REVM) The reason why it is removed is that CREATE2 is a opcode and can't be called outside of Revm. I can't think of easy way to do it. You can deploy it with CREATE and just change addresses after execution.

• migrate bug data from tinyevm

• figure out how to use the new inspector pattern

• Fix failed Rust tests:

  • tevm::revm_test test_bug_data_in_deploy
  • tevm::revm_test test_blockhash
  • tevm::revm_test test_deploy_with_args_and_value
  • tevm::revm_test test_deterministic_deploy
  • tevm::revm_test test_contract_method_revert
  • tevm::revm_test test_div_zero
  • tevm::revm_test test_deterministic_deploy_overwrite
  • tevm::revm_test test_call_forked_contract_from_local_contract
  • tevm::revm_test test_contract_deploy_transfer_query
  • tevm::revm_test test_get_set_balance
  • tevm::revm_test test_fork
  • tevm::revm_test test_call_trace
  • tevm::revm_test test_distance_signed
  • tevm::revm_test test_integer_truncation
  • tevm::revm_test test_exp_overflow
  • tevm::revm_test test_events
  • tevm::revm_test test_deadloop
  • tevm::revm_test test_reset_storage
  • tevm::revm_test test_overflow
  • tevm::revm_test test_revert_or_invalid
  • tevm::revm_test test_heuristics
  • tevm::revm_test test_seen_pcs
  • tevm::revm_test test_heuristics_signed_int
  • tevm::revm_test test_set_get_code
  • tevm::revm_test test_set_get_storage
  • tevm::revm_test test_peephole_optimized_if_equal
  • tevm::revm_test test_mod_zero
  • tevm::revm_test test_timestamp_and_block_number
  • tevm::revm_test test_runtime_configuration
  • tevm::revm_test test_seen_addresses
  • tevm::revm_test test_selfdestruct_and_create
  • tevm::revm_test test_sha3_mapping
  • tevm::revm_test test_tx_origin
  • tevm::revm_test test_tod
  • tevm::revm_test test_tx_origin_v2
  • tevm::revm_test test_gas_usage
  • tevm::revm_test test_library_method_with_large_string
  • tevm::revm_test test_sturdy_hack

• Fix Failed Tests in Redis Version

  • tevm::revm_test test_bug_data_in_deploy
  • tevm::revm_test test_blockhash
  • tevm::revm_test test_deploy_with_args_and_value
  • tevm::revm_test test_deterministic_deploy
  • tevm::revm_test test_contract_method_revert
  • tevm::revm_test test_div_zero
  • tevm::revm_test test_deterministic_deploy_overwrite
  • tevm::revm_test test_call_forked_contract_from_local_contract
  • tevm::revm_test test_contract_deploy_transfer_query
  • tevm::revm_test test_get_set_balance
  • tevm::revm_test test_fork
  • tevm::revm_test test_call_trace
  • tevm::revm_test test_distance_signed
  • tevm::revm_test test_integer_truncation
  • tevm::revm_test test_exp_overflow
  • tevm::revm_test test_events
  • tevm::revm_test test_deadloop
  • tevm::revm_test test_reset_storage
  • tevm::revm_test test_overflow
  • tevm::revm_test test_revert_or_invalid
  • tevm::revm_test test_heuristics
  • tevm::revm_test test_seen_pcs
  • tevm::revm_test test_heuristics_signed_int
  • tevm::revm_test test_set_get_code
  • tevm::revm_test test_set_get_storage
  • tevm::revm_test test_peephole_optimized_if_equal
  • tevm::revm_test test_mod_zero
  • tevm::revm_test test_timestamp_and_block_number
  • tevm::revm_test test_runtime_configuration
  • tevm::revm_test test_seen_addresses
  • tevm::revm_test test_selfdestruct_and_create
  • tevm::revm_test test_sha3_mapping
  • tevm::revm_test test_tx_origin
  • tevm::revm_test test_tod
  • tevm::revm_test test_tx_origin_v2
  • tevm::revm_test test_gas_usage
  • tevm::revm_test test_library_method_with_large_string
  • tevm::revm_test test_sturdy_hack

• Fix Python tests:

  • tests/test_account_snapshot.py::test_account_snapshot - RuntimeError: Invalid string length
  • tests/test_account_snapshot.py::test_redeploy - TypeError: argument 'deploy_to_address': 'NoneType' object cannot be converted to 'PyString'
  • tests/test_global_snapshot.py::test_global_snapshot[x_fns0-False-fastseq take and restore snapshot] - RuntimeError: Invalid string length
  • tests/test_global_snapshot.py::test_global_snapshot[x_fns1-False-slowseq take and restore snapshot] - RuntimeError: Invalid string length
  • tests/test_global_snapshot.py::test_global_snapshot[x_fns2-True-fastseq no snapshot] - RuntimeError: Invalid string length
  • tests/test_global_snapshot.py::test_global_snapshot[x_fns3-True-slowseq no snapshot] - RuntimeError: Invalid string length
  • tests/test_global_snapshot.py::test_compare_memory - RuntimeError: Invalid string length
  • tests/test_tinyevm.py::TestTinyEVM::test_deployment - TypeError: argument 'deploy_to_address': 'NoneType' object cannot be converted to 'PyString'
  • tests/test_tinyevm.py::TestTinyEVM::test_get_change_tx_gas_limit - AttributeError: 'builtins.TinyEVM' object has no attribute 'tx_gas_limit'
  • tests/test_tinyevm.py::TestTinyEVM::test_get_set_env_field - assert '"0x0"' == '0x0000000000...0000000000000'
  • tests/test_tinyevm.py::TestTinyEVM::test_instrument_config - AttributeError: 'builtins.REVMConfig' object has no attribute 'enabled'

• benchmark comparison with TinyEVM.

  • Python benchmarks from TinyEVM (from pytest)
    

  • Python benchmarks from TEVM (from pytest)
    

  • Rust benchmarks (from cargo bench), left side is from TEVM, right side is from TinyEVM. Test in which TEVM performs worse than TinyEVM are marked with red rectangle: cargo-bench-comparison.pdf

• Benchmark conclusion: while in most tests TEVM out performs TinyEVM, it performs worse at some tests:

  1. Python test_redeploy and significantly at test_account_snapshot (todo)
  2. Rust bench_call_tracing_with_different_executor and bench_contract_deploy_on_different_executors, this suggests TEVM is more expensive to create than TinyEVM