Adding regression tests requires too much work

[brianhuffman]

We have a test suite in the intTests directory, but its growth is hindered by how cumbersome it is to add tests. Adding a test requires creating a subdirectory with multiple files, including a shell script. I would like to have a directory where I could add a new test by simply creating a new .saw file (and possibly an expected-output file or other supporting files as necessary).

[ChrisEPhifer]

This is a pretty old ticket, but probably one of the items in the 0.6 milestone I'm best equipped to work on without a bunch of time spent staring at the code for the internals. I'm not sure I'll have something completely done by the end of next week (I'm taking the last two weeks of August off to move), but I'll take a crack at implementing something a little bit easier to work with than what we currently have. I'll likely need some assistance integrating whatever I come up with into the CI system, since I don't have much experience with CI/CD systems in general, but that's definitely not an immediate concern.

[kquick]

BTW, @ChrisEPhifer , I'm hoping to land #796 soon, which means the tests are run as part of a standard cabal test invocation. That should put it into scope for most CI scenarios without requiring extra work. This might let you use something like tasty-golden to extend the current mechanisms.

[ChrisEPhifer]

Ah, awesome! Thanks for pointing that out.

[brianhuffman]

My current habit is to use some existing saw-script tests as templates whenever I want to create a new regression test; that works well enough, so I'm closing the ticket.