-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce file driven tests. #125
Conversation
introduce file driven tests.
Like the idea, it probably a little more work for small tests, but as they get more compile it would be easier to add more tests for a rule. I tend to use |
Thanks. In other projects, I've found it encourages contributions. Folks seem to find it easier to just drop in a code sample, than to figure out the unit test nuances. I'll have to try -Dtest=name#subset with it. I haven't used that before -- seems like it could work. |
Also made above clearer it's |
fixed newline compare issues.
I don't think maven/surefire plays nice with running a single parameterized test. |
Introduce file driven tests.
I'm looking for feedback on this experimental one.
If you run the TestFiles unit test. It will pick up all .cf files under src\test\resources\com\cflint\tests. It applies the cflint config from the folder it is looking at, and compares the lint results with the expected results file.
Normally a sub folder would be dedicated to a particular checker, and the config only allows that particular rule to fire.
You can control the behavior of the run with test.properties. You normally care about only 1 test at a time when you are building a checker -- there is a option to run the test for only the most recently changed input file. There is also an option to overwrite the expected results instead of comparing against them.
https://cwe.mitre.org/data/definitions/482.html. This is revelant to CF. And was a good proof of concept to demonstrate the file test.