Feature & Refactor: Add Concurrency and Filler Skips to Improve tf Effienciency. Additional CLI arguments.`

[spencer-tb]

This PR improves our run time of tf from ~15s to ~5s.

So far I've added an additional function called parallel_fill that is equivalent to fill but with concurrency, as well as a --benchmark CLI argument that logs the run time of tf.

To test for yourself you will need to edit filler.fill() in main and switch between filler.parallel_fill(). Using tf --benchmark to log the times.

fill():

parallel_fill():

[marioevz]

Looks amazing!

Only a few comments.

Thanks!

[spencer-tb]

To summarise, we now have a refactor of ethereum_test_filling_tool:

• __init__.py
• filler.py
• main.py
• modules.py

With the following features:

• Default tf execution is concurrent/parallel. So far ~65% increase in speed. This should scale when more tests are added in the future.
• If a filler has already been filled into a fixture (.json file), it is skipped. Unless the filler file (the entire module not the test case) has been modified or the fixture file that is generated by the module does not exist in the output path (./fixtures by default).
  • This improves speed significantly. For example, now if I have ran tf for every filler, and then modify the withdrawals module withdrawals.py, tf will only fill this module. This takes a total time of ~1.76s.
• New CLI arguments:
  • tf --benchmark. Adds timing of filling to the output e.g: INFO:ethereum_test_filling_tool.filler:Filled test fixtures in 7.34 seconds.
  • tf --max-workers. Allows the user to set the number of workers used for concurrent tf execution. For serial execution (the previous way) use tf --max-workers 1.
  • tf --force-refill. Force tests to be filled. Works with --test-module and --test-categories.

[marioevz]

LGTM. Thanks!