strict mode for .out dir where it does not tolerate extra files and directories being generated

[faassen]

I was surprised to learn that trycmd does not check that all files and directories in .out dir are present and the same as the generated output, but only that if they are present they should be the same. This means that if a file or directory is missing from .out it succeeds even though I expected a failure.

Now I understand you might have the use case for ignoring extra files and directories, but it's still useful to have a mode where it's strict and doesn't tolerate these to exist. It's less of a problem if you use dump to generate .out, but sometimes I want to be able to adjust an .out exactly to my expectations without going through the dump route.

In addition, because git doesn't check in empty directories; if you want to detect in CI that you're relying on them, strict mode could help find that issue.

[Edit; I got this wrong first; it does detect missing files, just tolerates additional files]

[epage]

Personally, I feel like defaulting to tolerating extra files in .in is the right call as it encourages focusing on the subset of functionality being tested, causing less churn.

If we made this configurable, the ideal is per test. This can be done when using toml files but not when using trycmd files. That would require being globally configurable.