Test Cleanup #49
Merged
Test Cleanup #49
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NOTE: The following commit will fix all the path differences introduced
by this commit.
This rearrange has two goals:
1. make it easier to share code between python test case modules
2. eventually deduplicate test case data, so that guest/host separation
is no longer required.
Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
…eds testing Signed-off-by: Eric Richter <[email protected]>
This commit starts the process of factoring out some shared logic between the test cases for guest and host. This is an intermediate commit -- future commits WILL remove or rework code that may have been altered here. This is to avoid all the changes snowballing into one massive rework commit. That said, unfortunately a directory rename got wrapped up in this commit -- rather than implement a parameter that would probably be removed later, just rename the directory so that both match. Expect this to change again later. Anyway, the tl;dr is that `common.py` now defines a `SecvarctlTest` class for the unit tests to derive from, and for now factors out the command running and environment set-up logic. There will be more. Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Since the original valgrind runner has been replaced by ASAN, the old memcheck logic, args, etc can be removed. The getCmdResult() function can probably be merged with command(), unless there is a need for them to remain separate. Signed-off-by: Eric Richter <[email protected]>
…mdResult The getCmdResult function is always being using to check for a successful return code, so just make that function do the assert itself. This also allows for a custom error message that actually prints out the failing command, making debugging hopefully slightly easier? Signed-off-by: Eric Richter <[email protected]>
…ssertCmd*() call Signed-off-by: Eric Richter <[email protected]>
PEP8 is really picky. Addressed most of the complaints, though ignoring some. Fixed: - Use 4 spaces for indentation instead of tabs everywhere. - Related: fixed the absolutely weird mix of tabs and spaces in some spots - Removed all spaces between function name and () e.g. foo () -> foo() - added spaces around , where important - Replaced a needless string concat with f-string Ignored: - line length complaints. 80 column is archaic and impossible in python Signed-off-by: Eric Richter <[email protected]>
Not entirely sure why this is being assigned, so commenting out for now, in case I figure out that it is supposed to be used and that code was removed or something. Signed-off-by: Eric Richter <[email protected]>
Oh boy. - Fixed all indentation - Fixed all spaces around assignment: foo=1 -> foo = 1 - Fixed all comma spaces: foo,bar -> foo, bar - Fixed all comment spacing: bar #foo -> bar # foo - Removed some now unused code - Probably other changes too. Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
…True Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
…th assertTrue Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
…gument The host tests use a large table of commands and expected results, so rather than use some dumb reflection to call the right function, just create a function for those. Factor out the logic in assertCmdTrue and assertCmdFalse to this function, so those are now just simple wrappers to a single assert function. Signed-off-by: Eric Richter <[email protected]>
Signed-off-by: Eric Richter <[email protected]>
Each test file was very inconsistent about how it handled output logging, and also cause a lot of repeated code. Furthermore, it made debugging a failing test somewhat difficult, since the developer would have to figure out which log file would have the relevant output. This is also a significant concern for CI, since the output of the job will not display the relevant failure. This commit replaces all use of the logging with a new system that prints the output of a command if the related test case fails. This extends the current assertCmd family of functions to inject the output into the assert message. NOTE: this commit also removes the PPC-specific test case runners since that also had some custom logging behavior. If we want to keep PPC-specific test cases, they should be in a separate test file not included in the default runners. Signed-off-by: Eric Richter <[email protected]>
…ctor out shared globals to common Signed-off-by: Eric Richter <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
You know the drill, big block of commits changing things.
Major changes:
tests/directory<backend>/testdataswitched totestdata/<backend>, same with env, etccommon.pyassertCmd*family of functions, rather thanassertEqual(self.command(), ...)everywhereSome changes were made to the scripts that generate the testdata, but I didn't bother completing the full overhaul for them. I can either remove the few changes that made it in, or go back and make at least some of the changes needed (mostly path changes)