prepare run_tests to run multiple tests #1355
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello!
In this PR I've made some minor adjustments to allow for run_tests to be able to actually run a list of tests, instead of just one-at-a-time.
There are some issues that have arisen/will arise moving forward and I'd like to open a dialogue to see your opinion on how I should go about this.
Firstly, about
mocha.grep
...I see now the reason why mocha tests are being run with multiple calls to
before()
andafter()
hooks -- each call torun_tests
also calledmocha.grep
, turning a singular test suite containing 10 tests into 10 test suites containing 1 test. As my ultimate goal is to makemocha
tests run properly, I would have to either remove thegrep
call completely (this would mean ALL mocha tests in a file will always run, breaking "Run Selected Tests" functionality) or construct a longgrep
string containing all selected tests and pass that to thegrep
call.Second problem -- recording start and end times:
The approach by NTVS currently is to call
RecordStart
on the given test case, sendmocha
thetestName
andtestFile
and havemocha
grep the onetestName
out of the providedtestFile
. Then once thatprocess
exits, it's safe to callRecordEnd
and record the end time for the test, then repeat.However, I want to run a list of tests-- in order to do so, I have to provide
mocha
with the list up front so it can adjust its test suite accordingly. If I'm providingmocha
with a list of tests to run up front, we can no longer record accurate Start and End times for those tests. I could try to record the times inmocha.js
, but this adds another requirement for therun_tests
interface, and might make changes more difficult down the road.Any guidance is welcome and appreciated. Please let me know if you need me to elaborate on anything written here, hopefully I haven't rambled too much...
Thanks again