Write generators for all exercises

[petertseng]

In x-common, we have various exercises with canonical-data.json representing the recommended set of tests for that exercise. We created generators to more easily keep up with changes in x-common. Whenever x-common changes, the task is as simple as re-running the generator.

The current documentation is at https://github.com/exercism/xgo#generating-test-cases - if at any point we find that we should explain something better, we should add any necessary documentation.

Preliminaries:

• I find the list of our generators using ls exercises/*/cases_test.go | cut -d/ -f2 from the root of the xgo directory.
• I find the list of exercises with canonical-data.json using ls exercises/*/canonical-data.json | cut -d/ -f2 from the root of the x-common directory.

This issue concerns exercises that don't yet have a generator.

For each exercise, first we should decide whether we even want a generator.
An example where we don't want one is diamond, since property-based tests aren't represented in canonical-data.json.
So don't be afraid to suggest that we pass on having a generator if it creates more work than it saves.
After all, the point is to make maintenance of this track easy.

Or, we may decide to lazy-evaluate the generator:
If our current tests already match x-common, there is no current need to have a generator since it will make no difference currently.
So we may decide that we'll make a generator the next time the x-common JSON changes, and until then we'll stick with what we have.

If we decide we do want a generator for a given exercise:
First, we should check whether there are any cases that our track has that the common JSON doesn't have.
For each of those, we should consider the options:

• Propose adding it to the common JSON if we feel it can be valuable to other tracks
• Do nothing and just run the new generator (thus losing the extra case) if we feel the extra case does not contribute anything meaningful to this track
• Have our track's generator add the case after the JSON is parsed if we feel it adds value to the track but wouldn't add value to other tracks

• acronym ( acronym: create test case generator #691 )
• allergies ( allergies: add test generator #938 )
• all-your-base ( all-your-base: add test generator #945 )
• anagram
• atbash-cipher ( atbash-cipher: create test case generator #673 )
• beer-song
• binary Is deprecated in this track.
• binary-search ( binary-search: add test generator #957 )
• bracket-push ( bracket-push: create test case generator #672 )
• circular-buffer Warning: Stateful test.
• collatz-conjecture ( collatz-conjecture: add generator #958 )
• crypto-square Please note crypto-square: Remove test of intermediate functions problem-specifications#750 before generating.
• diamond Uses property-based tests, so won't be generated.
• difference-of-squares
• etl
• food-chain
• grains ( grains: add generator #959 )
• hello-world Will only ever have one test for foreseeable future, seems unnecessary to generate.
• isogram (isogram: add test generator #935)
• kindergarten-garden
• minesweeper
• nth-prime ( nth-prime: add generator #964 )
• nucleotide-count
• ocr-numbers
• pangram (keep in mind More test cases #873)
• pascals-triangle
• perfect-numbers ( perfect-numbers: create test case generator #952 )
• phone-number ( Phone number add generator #664 )
• pig-latin
• poker ( poker: create test case generator #686 )
• pov
• prime-factors
• queen-attack
• react Warning: Stateful test.
• run-length-encoding ( run-length-encoding: create test case generator #948 )
• robot-simulator
• saddle-points
• say ( say: generate tests #885 )
• scrabble-score ( scrabble-score: create test case generator #665 )
• secret-handshake ( secret-handshake: create test case generator #693 )
• sieve ( sieve: add generator #962 )
• sum-of-multiples ( sum-of-multiples: create test case generator #684 )
• tournament
• triangle Warning: Would significantly change exercise behaviour, due to Triangle: Update tests to check properties, not types problem-specifications#424.
• trinary Is deprecated in this track.
• variable-length-quantity ( variable-length-quantity: create test case generator #674 )
• word-search ( word-search: create test case generator #871 )
• wordy ( wordy: create test case generator #949 )

[petertseng]

The Ruby track takes on a similar project at exercism/ruby#396.

If, in the course of writing a generator, we find that there is significant duplicate code between generators that could be extracted into https://github.com/exercism/xgo/blob/master/gen/gen.go, we should do so, so that writing each generator is as little work as possible.

[robphoenix]

Please note the new exercise file structure for generators #610

[robphoenix]

The checklist in this issue is due to be replaced by separate issues.

Relevant comment by @petertseng from #611:

splitting #604 & #605 into separate issues

Y'all are right about the difficulty of keeping checklists up to date. We've seen it done both ways before in this repo too!

For #275 we split up into many different issues. I think this was when Blazon was new and I was very much still in that mindset.

In #414 and in #470 we had a checklist. Of these two, I was managing the checklist in #414 and not #470, but #414 is much smaller so I don't have much experience with that. I can say that when #414 was filed there were only eight exercises affected and I didn't want to set up filing the per-exercise issue for just eight of them. The recommendation is to use https://github.com/github/hub , I simply didn't have that set up (I used a modified version of Blazon for #275 ).

Which makes contributors' lives easier (Don't think only of ourselves)? It can be that the individual issues seem more tractable (harder for someone to know they need only take one item in a checklist). The disadvantage is if there's a chance of contributors seeing "Oh this project has 60 open issues they're obviously not very active" but perhaps that is not serious.

If agree on splitting them up, feel free to repurpose the #604 and #605 as the progress issue that the per-exercise issues link back to. I can't get to splitting them up until at least this weekend if not longer, so feel free to beat me to it. I'll say something if I start so that we don't duplicate work.

[robphoenix]

I'd like to suggest this issue is put on hold until #604 is closed. I'm worried that if this is also split into separate issues that it will create too many long running issues than is good for the repo. It also seems like a logical step to ensure our current generators are up to date before we start adding more.

[tleen]

@leenipper @robphoenix : May be worth throwing the experience into some documentation?

[stevejb71]

I'm using a template based approach for the OCaml generator. This has proved quite successful so far (30 out of 40 OCaml exercises have auto-generated tests, and also 1 Purescript exercise).

Templates are in here: https://github.com/exercism/xocaml/tree/master/tools/test-generator/templates. The language is a bit ad-hoc, I'm sure it could be improved.

[tleen]

Thanks @stevejb71 looks interesting, will check it out!

[tleen]

#803 is a temp fix to pangram till a generator is made for it.