Good news everyone: I've refactored the tests again! #7
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'd like to begin by thanking the various coffee shops of Cambridge, particularly their free Wi-Fi, for their contribution towards this pull request.
So I've refactored the tests again. What's changed this time?
The biggest new features are automatic test discovery and less dependence on environment variables. Having said that, I haven't been able to eliminate
COCOTB_TOP
andRTL_TOP
entirely. Let's start with the good news: the call syntax for the cocotb_test_wrapped has been significantly streamlined to make writing tests easier. Take a test case for my decoder as an example:We can restrict the source files globbed by the compiler with the
src
argument, used here to stop compilation errors with unrelated code from failing this test suite (hint: this is generally a good idea if you can make it work with your tests).src_dirs
is defined outside the function scope because all the tests in this file use the same source files. Not passing an argument here compiles the wholehdl/rtl/src
directory. In addition, you can provide a custom include path with theinclude
argument; by default, this points tohdl/rtl/include
, where I've placed some of our existing files as new header files.The
toplevel
arg is needed to tell both the compiler and runner which module is our top. The duplication there annoysme and makes my original plan of sharing a single object between all tests in a module impossible (for now). I'd quite like to pursue this in future because we're compiling the same code many times across any given test suite, so I'll investigate if something like
'-'
or specifying multiple top level modules could fix this issue.test_search_path
is new, and, uses glob to recursively find all files namedcocotb_test_*.py
from within the specified directory. This allows us to write multi-file test suites for modules. However, we already have a lot of flexibility across multiple pytests within multiple test_*.py files, so we should review this approach.I've added some other args to control flags etc, but I doubt we'll be using them much
I've hacked this all together to work more robustly with the GitHub Actions scripts too, so that's all looking good.
Unfortunately, there are still a few drawbacks to this approach:
export RTL_TOP=
etc commands to get the right paths for compilation (RTL_TOP
) and saving the results (COCOTB_TOP
). I've got rid of the GitHub script and just moved the commands intoverif.yml
, which works well enough. The benefit of this approach is that we can specify entirely relative paths, and only have to worry about doing any path logic in thecoco_wrapper
module.Before we merge this PR, let's make sure all these improvements are copied to issues or that project board so I remember to actually make the changes. Same with the last PR.