This repository has been archived by the owner on Jan 6, 2025. It is now read-only.
Workflow improvements: helpers pkg with suite generator base, and automatic makefile targets #31
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
protolambda
changed the title
|
Streamlined it some more. Everyone can easily write and work on new test-generators now 🎉 |
|
Awesome work :) |
jannikluhn
approved these changes
Mar 25, 2019
jannikluhn
reviewed
Mar 26, 2019
Co-Authored-By: protolambda <[email protected]>
|
Implemented in ethereum/consensus-specs#851 |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To speed up and ease the creation of new test generators, I implemented a base package which makes it really easy to create new generators with. The format conforms to my proposed suite format in #29
I also updated the makefile, as most people hate writing makefiles, when they could be spending time on new test-generators/research.
The idea is basically that top-level folder names define the test-types. In these folders, there's a bunch of yaml files, which we call "suites". Each suite has a name, summary, constants-config, fork, and a test-cases list.
The goal of a generator is to a set of suites that cover different facets of the spec-topic (test type) being tested.
How generators look like
This PR included just the helper package at first, but since it's more about workflow, I decided to make it streamlined: people should not have to touch the makefile, just write a test-generator. And with the helpers package, this should be super easy.
Check the new README for an example on how to write a new test generator.
WIP
Work in progress. What we need to get this running:
Note that this does not break any existing generators. Although I do hope existing generators move to adopt this in the future. SSZ is the easiest to port, see below. Note that the existing generators are not exactly conformaing to my proposed suite header data, but still work, and output still in the same way.
Credits
Much of the generator code has been adapted from the previous SSZ test generator. The object yield pattern is very nice, and command-line arguments are great to have too.