TestReporter does not capture the correct TestIdentifier when used with DynamicTests

[nilsga]

I'm investigating JUnit5 for a project, and I find the concept of TestFactory and DynamicTests interesting for our use case. During the execution of a DynamicTest, I would also like to report some additional information for that particular test, so I added a TestReporter parameter to the TestFactory method:

@TestFactory
public void createTests(TestReporter reporter) {
  Stream<DynamicTest> tests = createTests(reporter);
}

Using a TestExecutionListener, I can see that the ReportEntrys are published, but they are all associated with the wrong TestIdentifier. Quite obvious, when thinking about it, since the TestReporter, once injected, is associated with the ExtensionContext it was assigned when it was created.

So it does not seem to be possible to use a TestReporter in conjunction with DynamicTest this way. Is it possible to achieve this some other way? Since the TestExecutionListener reports the correct TestIdentifier for lifecycle events (e.g. executionFinished) for the other DynamicTests, it seems like it should be possible somehow to dynamically assign the correct context to a TestReporter.

Related Issues

• Introduce support for parameterized tests #14
• Enhance support for dynamic tests #378
• Improve documentation of DynamicTest lifecycle #386

[sbrannen]

FYI: this is related to #378 and #386.

[sbrannen]

So it does not seem to be possible to use a TestReporter in conjunction with DynamicTest this way.

That's correct.

Is it possible to achieve this some other way?

No, it is unfortunately currently not possible.

Please see the discussions in #378 and #386 for further details.

[nilsga]

Yes, found those issues after posting this. Thanks. Sorry about the duplication.

[sbrannen]

No worries!

The use of a TestReporter in dynamic tests will be addressed in #378, so I'll close this issue as a duplicate now.

[whytox]

One simple solution would be to save the dynamicTest's TestIdentifier in a variable of the TestExecutionListener when the testExecutionStarted lifecycle method kicks and use that instead of the TestIdentifier supplied in the arguments of reportEntryPublished.

So, for example, supposing that your dynamic test has access to the TestFactory's TestReport and use it to publishEntries:

public class MyReportListener extends SummaryGeneratingListener {

     private TestIdentifier currentTestBeingExecuted;

    @Override
    public void executionStarted(TestIdentifier testIdentifier) {
        super.executionStarted(testIdentifier);
        if (testIdentifier.isContainer()) {
            return;
        }
        currentTestBeingExecuted = testIdentifier;
    }


    @Override
    public void reportingEntryPublished(TestIdentifier testIdentifier, ReportEntry entry) {
        super.reportingEntryPublished(testIdentifier, entry);
        TestIdentifier dynamicTestIdentifier = testIdentifier;
        if (!testIdentifier.equals(currentTestBeingExecuted)) {
            // then the entry has been probably been published by a dynamic test
            // using its container TestReporter
            dynamicTestIdentifier = currentTestBeingExecuted;
         }
        // do your stuff with the real testIdentifier
    }

[marcphilipp]

Here's another workaround: #4175 (reply in thread)

[marcphilipp]

Duplicate of #378