Should we add x-common versions to our non-generated exercises?

[petertseng]

Putting the x-common version in all of our generators gives us a benefit: We can easily detect when our tests go out of date, so we know when to rerun the generators.

Until we finish #605, we have non-generated exercises. Let's say #605 will take a few months to complete, and/or we decide that there are some exercises we don't want to generate (how to generate react???).

Would it help if we added an x-common version for all currently non-generated exercises? That would help us determine which non-generated exercises are out of date. That can help:

• Prioritise which exercises most urgently need a generator (which will automatically get them up to date)
• Or maybe signal to us that if we don't think we will get a generator soon, at least maybe we can evaluate the x-common change and see if it's worth it to add the new tests by hand until we get a generator.

If the xgo tests are a superset of x-common for some version of x-common, there will be comment "// x-common version: x.y.z" just like in our generated files, and also note which tests are xgo-specific.

Otherwise, there will be the comment "// x-common version: none". The presence of the comment shows that it's been looked at but no suitable x-common version was found. (Or is it OK not to have the comment at all?)

If y'all think this is a good idea, I volunteer to do it.

[leenipper]

Sounds helpful. We can add a script in xgo/bin which does a report on generator vs. canonical versions to get a summary.

So, IIUC, the new comment will be located in the .meta/gen.go of each exercise, and the version x.y.z will agree with the x-common version that is shown in the cases_test.go which was "just generated" or previously generated. Correct ?

xgo tests are a superset of x-common for some version of x-common, there will be comment "// x-common version: x.y.z" just like in our generated files

I am not quite getting the "superset" idea here. If the comment said "// x-common version: x.y.z + additional tests; see below", then it makes sense to me. Meaning that the original tests were from x-common x.y.z but more were added manually in different test case data.

[petertseng]

So, IIUC, the new comment will be located in the .meta/gen.go of each exercise, and the version x.y.z will agree with the x-common version that is shown in the cases_test.go which was "just generated" or previously generated. Correct ?

Yeah something like that! This is already happening for the generated exercises. This proposal just says I'll add them by hand to the non-generated exercises, until they get a generator.

I am not quite getting the "superset" idea here. If the comment said "// x-common version: x.y.z + additional tests; see below", then it makes sense to me.

Ah, yeah, that could be done. In my mind the purpose of marking the xgo-specific tests is just to make sure that when we create a generator that we consider whether we want to delete the test or make sure they're retained even after generation. If we'd rather wait to perform the check, I suppose I don't need to mark them.

[leenipper]

This is already happening for the generated exercises

Oh, you must mean the x-common version comment in the generated cases_test.go.

add them by hand to the non-generated exercises, until they get a generator

Oh, in that case .meta/gen.go won't exist yet.
So, what file is best to place the x-common comment inside in this case ?

make sure they're retained even after generation

So, the added tests are placed such that re-generation retains both the canonical tests and the added ones.

[petertseng]

This is already happening for the generated exercises

Oh, you must mean the x-common version comment in the generated cases_test.go.

👍

add them by hand to the non-generated exercises, until they get a generator

Oh, in that case .meta/gen.go won't exist yet.
So, what file is best to place the x-common comment inside in this case ?

I'd say to place them near the test cases in the*_test.go file. This way we will very likely remember to remove the comment (to avoid duplicate information) when we make a generator.

So, the added tests are placed such that re-generation retains both the canonical tests and the added ones.

Yeah, if we decide we want to keep some of them. alternatives of course are to no longer have them (match x-common) or to get them accepted to x-common.

[leenipper]

Very good. Understand the approach now.

[petertseng]

For planning:

The list of such unversioned exercises is:

                   acronym: 0.0.0 -> 1.0.0
             all-your-base: 0.0.0 -> 1.0.0
                 allergies: 0.0.0 -> 1.0.0
                   anagram: 0.0.0 -> 1.0.1
             atbash-cipher: 0.0.0 -> 1.0.0
                 beer-song: 0.0.0 -> 1.0.0
             binary-search: 0.0.0 -> 1.0.0
              bracket-push: 0.0.0 -> 1.1.0
           circular-buffer: 0.0.0 -> 1.0.0
             crypto-square: 0.0.0 -> 2.0.0
                   diamond: 0.0.0 -> 1.0.0
     difference-of-squares: 0.0.0 -> 1.0.0
                       etl: 0.0.0 -> 1.0.0
                food-chain: 0.0.0 -> 1.0.0
                    grains: 0.0.0 -> 1.0.0
                   isogram: 0.0.0 -> 1.1.0
       kindergarten-garden: 0.0.0 -> 1.0.0
               minesweeper: 0.0.0 -> 1.0.0
                 nth-prime: 0.0.0 -> 1.0.0
          nucleotide-count: 0.0.0 -> 1.0.0
               ocr-numbers: 0.0.0 -> 1.0.0
                   pangram: 0.0.0 -> 1.0.0
          pascals-triangle: 0.0.0 -> 1.0.0
           perfect-numbers: 0.0.0 -> 1.0.1
                 pig-latin: 0.0.0 -> 1.0.0
                       pov: 0.0.0 -> 1.1.1
             prime-factors: 0.0.0 -> 1.0.0
              queen-attack: 0.0.0 -> 1.0.0
                     react: 0.0.0 -> 1.0.0
           robot-simulator: 0.0.0 -> 1.0.0
             saddle-points: 0.0.0 -> 1.0.0
                       say: 0.0.0 -> 1.0.0
          secret-handshake: 0.0.0 -> 1.0.0
                     sieve: 0.0.0 -> 1.0.0
          sum-of-multiples: 0.0.0 -> 1.0.0
                tournament: 0.0.0 -> 1.3.0
                  triangle: 0.0.0 -> 1.0.0
  variable-length-quantity: 0.0.0 -> 1.0.0
               word-search: 0.0.0 -> 1.0.0
                     wordy: 0.0.0 -> 1.0.0

[petertseng]

I've decided it's not necessary to put these (#670, #671) in a student visible place. The goal of this was to tell us which exercises are most out of date and therefore would benefit the most from a generator. This can be achieved by simply reporting the information I found.

Our up-to-date exercises:

             all-your-base: 1.0.0 == 1.0.0
                       bob: 1.0.0 == 1.0.0
                   bowling: 1.0.0 == 1.0.0
                    change: 1.0.0 == 1.0.0
                   connect: 1.0.0 == 1.0.0
                custom-set: 1.0.1 == 1.0.1
                       etl: 1.0.0 == 1.0.0
                food-chain: 1.0.0 == 1.0.0
                     forth: 1.0.0 == 1.0.0
                gigasecond: 1.0.0 == 1.0.0
                    grains: 1.0.0 == 1.0.0
                   hamming: 1.0.0 == 1.0.0
    largest-series-product: 1.0.0 == 1.0.0
                      leap: 1.0.0 == 1.0.0
                      luhn: 1.0.0 == 1.0.0
                    meetup: 1.0.0 == 1.0.0
                 nth-prime: 1.0.0 == 1.0.0
                 raindrops: 1.0.0 == 1.0.0
         rna-transcription: 1.0.0 == 1.0.0
            roman-numerals: 1.0.0 == 1.0.0
            scrabble-score: 1.0.0 == 1.0.0
                 transpose: 1.0.0 == 1.0.0
                word-count: 1.0.0 == 1.0.0
                     wordy: 1.0.0 == 1.0.0

All other exercises, taking into account what I found in #670 and #671:

                   acronym: 0.9.0.20151202 -> 1.0.0
                 allergies: 0.8.0 -> 1.0.0
                   anagram: 0.6.0 -> 1.0.1
             atbash-cipher: 0.9.0 -> 1.0.0
                 beer-song: 0.0.0 -> 1.0.0
                    binary: DEPRECATED
             binary-search: 0.0.0 -> 1.0.0
              bracket-push: 0.0.0 -> 1.1.0
           circular-buffer: 0.0.0 -> 1.0.0
                     clock: 1.0.0 -> 1.0.1
                   counter: DEPRECATED
             crypto-square: 0.0.0 -> 2.0.0
                   diamond: IGNORED: property-based
     difference-of-squares: 0.9.0 -> 1.0.0
               hello-world: IGNORED: no version
               hexadecimal: DEPRECATED
                   isogram: 0.8.0.20160608 -> 1.1.0
       kindergarten-garden: 0.0.0 -> 1.0.0
               minesweeper: 0.0.0 -> 1.0.0
          nucleotide-count: 0.9.0 -> 1.0.0
               ocr-numbers: 0.0.0 -> 1.0.0
                     octal: DEPRECATED
                   pangram: 0.8.0.20160416 -> 1.0.0
          pascals-triangle: 0.0.0 -> 1.0.0
           perfect-numbers: 0.2.0 -> 1.0.1
              phone-number: 1.1.0 -> 1.2.0
                 pig-latin: 0.5.0 -> 1.0.0
                       pov: 0.0.0 -> 1.1.1
             prime-factors: 0.4.0 -> 1.0.0
              queen-attack: 0.0.0 -> 1.0.0
                     react: 0.0.0 -> 1.0.0
           robot-simulator: 0.0.0 -> 1.0.0
             saddle-points: 0.3.0 -> 1.0.0
                       say: 0.7.0 -> 1.0.0
          secret-handshake: 0.6.0 -> 1.0.0
                     sieve: 0.7.0 -> 1.0.0
          sum-of-multiples: 0.5.0 -> 1.0.0
                tournament: 0.0.0 -> 1.3.0
                  triangle: 0.0.0 -> 1.0.0
                   trinary: DEPRECATED
  variable-length-quantity: 0.0.0 -> 1.0.0
               word-search: 0.0.0 -> 1.0.0

[petertseng]

Since the goal (how up-to-date are we?) is achieved, I can close this now. Feel free to continue commenting if you have extra information to about how up-to-date we are.