book-store: canonical-data.json needs optimization for automatic test generation

[rpottsoh]

I have been monitoring the recent issues regarding the canonical data for a few of the exercises and it has me thinking more about the canonical data for this exercise.

While I think it is fine for human consumption I think it will not be liked by automated test writers.

1. Test decriptions are too long.
2. Test number 8 while also being too wordy also provides a hint towards the solution. I don't much care for this. The "targetgrouping" should be enough to provide the track maintainer with the information they need to write an example solution. The "expected" amount is what is being tested for.

I would appreciate any suggestions that would help me cut down the wordiness to help facilitate automatic test generation and also your thoughts regarding the eighth test. Any feedback from those utilizing automatic test generation would be helpful.

Thanks

[robkeim]

As you mentioned @rpottsoh the descriptions are helpful for a human reading it but not meant to be used for the real test name.

I don't use any test generation, but I was helped by the descriptions when implenting the C# and F# tracks. I cut the names down to a shorter length and for the 8th I left out everything hinting at the solution.

One approach might be to have a "name" field for the suggested test name which could be read by auto test generators and then a "notes/info" section which could contain more detailed human readable info.

[rpottsoh]

@robkeim my only issue with this is that I believe those tracks that have automated test generation likely are already using the "description" field for test names. My JSON skills are not that strong. Are comments permitted that would be ignored by automation? What is now the "description" could become a comment and the "description" something shorter that would be better suited for a test name.

[kotp]

Or leave the "description" field as it is, and add a "testname" key. The changes required in the generator would be a one time deal.

[rpottsoh]

@kotp that is essentially what @robkeim is saying. I was just concerned that some standard was already in affect where "description" had to be, or is expected to be, the test name.

[robkeim]

@rpottsoh if you want to avoid a potential breaking change, then keep the existing "description" field for the test name and add another field called "detailedDescription" which can be used for the humans implementing the tests in tracks where it's done manually.

[rpottsoh]

This seems like a reasonable compromise. What do you think @kotp?

[kotp]

@rpottsoh yes, we are using the description as the test name for Ruby, and I think this is as good a time as any to talk about changing the canonical to have a key for test names that can be used.

/cc @exercism/track-maintainers

[masters3d]

+1 on test names.

[Insti]

description should be the description. If there needs to be a separate test name property then add a test_name property. It's not a problem if generators need to change.

But I think that in general people's descriptions are overly verbose, and if they wrote better tests and descriptions this would not be necessary. I am worried that we will end up with the same problem again where the test name is useless and the description is still verbose and doesn't say why the test is useful.

eg: DON'T DO THIS

            "test_name": "test_1_1_2_2_3_3_4_4_5_5_1_2",
            "description": [
                            "A basket containing twelve books consisting of ",
                            "three copies of the first two books, plus two ",
                            "each of the remaining three books in the series."
           ]

[rbasso]

I agree that description should be short, but I'm still not understanding what is the need for another key-value pair.

Isn't description already the string that describes the test to the user?!

I think that in all the exercises in the Haskell track we used the descriptions directly, without changing a single character, and we are not even using automated test generators...

Am I missing something here?!

[rpottsoh]

I have taken some time and reworked all the descriptions. Please review #514.

[robkeim]

@rpottsoh I just had a look through the changes and left my feedback.

[rpottsoh]

I certainly plan to fix the formatting. Tried an online JSON editor and that's what it did to the file. :(

[kotp]

#514 fixes this issue