Interfaces implemented by test classes are not registered for reflection

[sbrannen]

Overview

Given the following interface:

package org.example;

import java.util.List;

interface TestInterface {

	static List<String> names() {
		return List.of("Sarah", "Susan");
	}
}

... and the following test class:

package org.example;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;

class DemoTests implements TestInterface {

	@ParameterizedTest
	@MethodSource("names")
	void test(String name) {
		assertEquals(5, name.length());
		assertTrue(name.startsWith("S"));
	}
}

... DemoTests passes on the JVM but fails within a native image as follows.

Failures (1):
  JUnit Jupiter:DemoTests:test(String)
    MethodSource [className = 'org.example.DemoTests', methodName = 'test', methodParameterTypes = 'java.lang.String']
    => org.junit.platform.commons.PreconditionViolationException: Could not find factory method [names] in class [org.example.DemoTests]

However, if TestInterface were a class that DemoTests extended the test would then pass.

The reason is that the JUnitPlatformFeature currently invokes registerAllClassMembersForReflection() for all classes with the test class hierarchy, but that recursive algorithm ignores implemented interfaces.

native-build-tools/common/junit-platform-native/src/main/java/org/graalvm/junit/platform/JUnitPlatformFeature.java

Lines 150 to 158 in 979e9ec

	private void registerTestClassForReflection(Class<?> clazz) {
	debug("Registering test class for reflection: %s", clazz.getName());
	nativeImageConfigImpl.registerAllClassMembersForReflection(clazz);
	forEachProvider(p -> p.onTestClassRegistered(clazz, nativeImageConfigImpl));
	Class<?> superClass = clazz.getSuperclass();
	if (superClass != null && superClass != Object.class) {
	registerTestClassForReflection(superClass);
	}
	}

The registerTestClassForReflection() method in JUnitPlatformFeature should therefore be revised to process implemented interfaces (and super-interfaces) at each level of the class hierarchy.

Related Issues

• @ParameterizedTest with @MethodSource are not supported.  #51
• Configure types referenced via @ExtendWith, @ArgumentsSource, etc. for reflection  #54
• Add support for @FieldSource for @ParameterizedTest methods #638

[vjovanov]

Thanks for reporting, this is on our short-term plan and we will look into it. MethodSource seems to be computed at run time and needs extra metadata.

Our future implementation of the JUnit feature will have to register this metadata based on the test annotations.

[sbrannen]

Hi @vjovanov,

MethodSource seems to be computed at run time and needs extra metadata.

I apologize: my use of @MethodSource to demonstrate the larger issue was perhaps a bit misleading.

NBT already has specific support for registering reflection metadata for fully-qualified method names configured via @MethodSource, which can be seen here:

native-build-tools/common/junit-platform-native/src/main/java/org/graalvm/junit/platform/config/jupiter/JupiterConfigProvider.java

Lines 144 to 166 in 979e9ec

	private static Class<?>[] handleMethodReference(String... methodNames) {
	List<Class<?>> classList = new ArrayList<>();
	for (String methodName : methodNames) {
	String[] parts = methodName.split("#");
	/*
	* If the method used as an argument source resides in a different class than the test class, it must be specified
	* by the fully qualified class name, followed by a # and the method name
	*/
	debug("Found method reference: %s", methodName);
	if (parts.length == 2) {
	String className = parts[0];
	debug("Processing method reference from another class: %s", className);
	try {
	classList.add(Class.forName(className));
	} catch (ClassNotFoundException e) {
	debug("Failed to register method reference for reflection: %s Reason: %s", className, e);
	}
	} else {
	debug("Skipping method reference as it originates in the same class as the test: %s", methodName);
	}
	}
	return classList.toArray(new Class<?>[0]);
	}

If the method name is not fully-qualified, that simply logs a debug message:

debug("Skipping method reference as it originates in the same class as the test: %s", methodName);

That "same class" part actually applies to the test class hierarchy; however, it should apply to the entire type hierarchy.

Hence, the following passes both on the JVM and within a native image, but my original example with class DemoTests implements TestInterface does not pass within a native image.

class BaseTests {

	static List<String> names() {
		return List.of("Sarah", "Susan");
	}
}

class DemoTests extends BaseTests {

	@ParameterizedTest
	@MethodSource("names")
	void test(String name) {
		assertEquals(5, name.length());
		assertTrue(name.startsWith("S"));
	}
}

The larger issue is that "test interfaces" are ignored in JUnitPlatformFeature. This means that any use case relying on reflection for fields, methods, etc. from test interfaces will not work without custom metadata configuration.

See also: Test Interfaces and Default Methods in the JUnit 5 User Guide.

Our future implementation of the JUnit feature will have to register this metadata based on the test annotations.

As I mentioned above, that is already covered by the JupiterConfigProvider.

The fix for this issue should likely be as simple as modifying the registerTestClassForReflection() method in JUnitPlatformFeature so that it recursively processes the entire type hierarchy (including implemented interfaces) for the test class.

If you need any further clarification, just let me know.

Thanks,

Sam