Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discover test configuration on enclosing class for nested test class [SPR-15366] #19930

Closed
13 tasks done
spring-projects-issues opened this issue Mar 20, 2017 · 31 comments
Closed
13 tasks done
Assignees
Labels
in: test Issues in the test module type: enhancement A general enhancement
Milestone

Comments

@spring-projects-issues
Copy link
Collaborator

spring-projects-issues commented Mar 20, 2017

Sam Brannen opened SPR-15366 and commented

Status Quo

Spring's support for JUnit Jupiter already supports detection of test configuration (e.g., (@ContextConfiguration) on @Nested classes.

However, if a @Nested class does not declare its own test configuration, Spring will not find the configuration from the enclosing class.

See also this discussion on Stack Overflow regarding nested test classes and @Transactional.

Proposal

Inspired by issue 8 from the spring-test-junit5 project, it would perhaps be desirable if the Spring TestContext Framework would discover test configuration on an enclosing class for a @Nested test class.

Deliverables


Affects: 5.0

Issue Links:

6 votes, 10 watchers

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

Current work is being performed in the following branch:

https://github.com/sbrannen/spring-framework/tree/SPR-15366

@spring-projects-issues
Copy link
Collaborator Author

Andy Wilkinson commented

Sam asked me to comment on this with reference to this Spring Boot issue. In looking at that issue I observed that the following test would start two separate application contexts:

@ExtendWith(SpringExtension.class)
@SpringBootTest
public class TwoDifferentContextsTests {

	@Autowired
	ApplicationContext context;

	@Nested
	class NestedTests {

		@Test
		public void test() {
		}

	}

}

One context is started for TwoDifferentContextsTests. Due to @SpringBootTest, it finds the @SpringBootApplication-annotated class and uses that as the basis for the context's configuration. A second context is started for NestedTests as it hasn't inherited @SpringBootTest from the enclosing class and, therefore, has different configuration. In fact, NestedTests doesn't really have any configuration. It's only able to have a context created for it due to the context customiser factories declared by spring-boot-test in spring.factories. It enters the if block rather than throwing in this code in AbstractDelegatingSmartContextLoader:

// If neither of the candidates supports the mergedConfig based on resources but
// ACIs or customizers were declared, then delegate to the annotation config
// loader.
if (!mergedConfig.getContextInitializerClasses().isEmpty() || !mergedConfig.getContextCustomizers().isEmpty()) {
	return delegateLoading(getAnnotationConfigLoader(), mergedConfig);
}

// else...
throw new IllegalStateException(String.format(
		"Neither %s nor %s was able to load an ApplicationContext from %s.", name(getXmlLoader()),
		name(getAnnotationConfigLoader()), mergedConfig));

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

I actually assumed it was due to an auto-registered feature of Spring Boot Test (i.e., something like a ContextCustomizer).

So thank you for confirming my suspicions!

@spring-projects-issues
Copy link
Collaborator Author

Stefan Ludwig commented

FYI: I've extended the original bug example from the (GitHub issue) with "Boot - less" implementations of @Nested Spring REST Docs tests:
https://github.com/slu-it/bug-spring-restdocs-nested-tests/tree/master/src/test/java/withoutboot

The following test fails because the WebApplicationContext could not be provided by the Spring Extension (not found in context):

@SpringJUnitWebConfig(classes = Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    @BeforeEach
    void setup(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Stacktrace;

org.junit.jupiter.api.extension.ParameterResolutionException: Failed to resolve parameter [org.springframework.web.context.WebApplicationContext arg0] in executable [void withoutboot.FailingTest.setup(org.springframework.web.context.WebApplicationContext,org.springframework.restdocs.RestDocumentationContextProvider)]

	at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameter(ExecutableInvoker.java:221)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameters(ExecutableInvoker.java:174)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameters(ExecutableInvoker.java:135)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:116)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.invokeMethodInExtensionContext(ClassTestDescriptor.java:302)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$12(ClassTestDescriptor.java:290)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:135)
	at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:155)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:134)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:109)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:58)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:112)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:55)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:43)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:65)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.web.context.WebApplicationContext' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:1509)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1104)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1065)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.resolveDependency(AbstractAutowireCapableBeanFactory.java:344)
	at org.springframework.test.context.junit.jupiter.ParameterAutowireUtils.resolveDependency(ParameterAutowireUtils.java:98)
	at org.springframework.test.context.junit.jupiter.SpringExtension.resolveParameter(SpringExtension.java:177)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameter(ExecutableInvoker.java:207)
	... 69 more

If the context configuration is duplicated by adding @SpringJUnitWebConfig(classes = Application.class) on the @Nested class it works (as previously mentioned by Andy Wilkinson). But this doesn't feel very intuitive.

(comment moved from SPR-16595)

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

Stefan Ludwig,

I think the behavior you've described might actually be a bug in JUnit Jupiter instead of Spring.

Can you please tell me what happens if you move the setup for MockMvc into a constructor as follows?

@SpringJUnitWebConfig(Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    FailingTest(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Thanks in advance for feedback!

@spring-projects-issues
Copy link
Collaborator Author

Stefan Ludwig commented

If I do that, it works!
I added that code as an working example to the repository.

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

If I do that, it works!

Awesome... and depressing... at the same time. ;-)

Awesome that it works for you!

Depressing (in a facetious way) for me because that means it is in fact an issue in JUnit Jupiter's handling of @BeforeEach methods with regard to the ExtensionContext supplied to a ParameterResolver registered for an enclosing test class when executing a test method within a @Nested test class.

And if that's too much of a mouthful for you, don't worry: I'll address the issue you've described within JUnit Jupiter.

Cheers,

Sam

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Mar 17, 2018

Stefan Ludwig commented

Glad to have helped! And sorry to depress you ;)

But are you sure it's not Springs management / caching of application contexts?

I thought it might be the same issue as, or at least very closely related to, #21136 because when I debug the following example:

@SpringJUnitWebConfig(classes = Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    @BeforeEach
    void setup(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Test
    void testAndDocumentation() throws Exception {
        mockMvc.perform(get("/hello"))
            .andExpect(status().is2xxSuccessful())
            .andExpect(jsonPath("message", equalTo("Hello World!")))
            .andDo(document("hello-get-200"));
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Adding a breakpoint in the SpringExtension within public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext).
I get an GenericWebApplicationContext with all the expected beans for the main test class.
But for the @Nested class I just get a GenericApplicationContext with none of the expected beans in it.

That sounds to me like what was described in #21136. Especially because adding @SpringJUnitWebConfig(classes = Application.class) to the @Nested class actually makes the test work and for both contexts (nested and enclosing class) the same application context is used in the parameter resolver.

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

FYI: I have opened the following issue for JUnit Jupiter.

junit-team/junit5#1332

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

Adding a breakpoint in the SpringExtension within public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext).
I get an GenericWebApplicationContext with all the expected beans for the main test class.
But for the @Nested class I just get a GenericApplicationContext with none of the expected beans in it.

Well, you have spring-boot-test in the classpath -- right?

Otherwise, Spring will not load any ApplicationContext for the @Nested class (if the nested class does not provide its own annotations).

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Mar 17, 2018

Sam Brannen commented

That sounds to me like what was described in #21136.

No. #21136 covers use cases where @Import is used on a test class.

@spring-projects-issues
Copy link
Collaborator Author

Stefan Ludwig commented

Oh, I did not think about the spring-boot-test dependency existing influencing the test. Thanks for clarifying!

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

You're welcome.

And... yeah... I agree that having spring-boot-test change things just by being present in the classpath can be a bit confusing at times. ;-)

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

I added that code as an working example to the repository.

Speaking of which, thanks for creating that!

It's a nice collection of examples.

@spring-projects-issues
Copy link
Collaborator Author

Andy Wilkinson commented

having spring-boot-test change things just by being present in the classpath

It doesn’t. You have to using SpringRunner or SpringExtension too. At that point we’re at the mercy of the registration mechanism for ContextCustomizerFactory and the like. Is there a way to opt those back out again?

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

I of course meant "change things for tests executing with the Spring TestContext Framework".

Sorry if that was not apparent.

@spring-projects-issues
Copy link
Collaborator Author

Sam Brannen commented

Is there a way to opt those back out again?

No, there is currently no built-in mechanism for disabling a ContextCustomizer that was registered automatically.

Maybe we should introduce something analogous to Boot's @EnableAutoConfiguration(exclude = ...) support.

@spring-projects-issues
Copy link
Collaborator Author

@sdeleuze
Copy link
Contributor

Hey @sbrannen, I got several developers (including the awesome @jnizet) asking for a fix for spring-projects/spring-boot#12470 which depend on that issues, any chance you could fix it for 5.2 in order to allow Boot team to support correctly nested classes in 2.2?

@sbrannen
Copy link
Member

@sdeleuze, I'll do my best to prioritize this one.

@sbrannen sbrannen modified the milestones: 5.x Backlog, 5.2 M1 Feb 15, 2019
@sbrannen
Copy link
Member

Now tentatively slated for 5.2 M1.

@sbrannen sbrannen removed this from the 5.2 M1 milestone Mar 18, 2019
sbrannen added a commit that referenced this issue Oct 9, 2020
sbrannen added a commit that referenced this issue Oct 9, 2020
Use MetaAnnotationUtils.findMergedAnnotation() instead of the
MergedAnnotations API in order to provide proper support for
@NestedTestConfiguration.

See gh-19930
sbrannen added a commit that referenced this issue Oct 10, 2020
Use MetaAnnotationUtils.findMergedAnnotation() instead of the
MergedAnnotations API in order to provide proper support for
@NestedTestConfiguration.

See gh-19930
sbrannen added a commit that referenced this issue Oct 12, 2020
This commit also includes several experiments that will be later
deleted. These will remain here in the commit history so that they do
not get lost in case parts of them are needed at a later date.

See gh-19930
sbrannen added a commit that referenced this issue Oct 12, 2020
@sbrannen
Copy link
Member

Reopening to address additional deliverables.

@sbrannen sbrannen reopened this Oct 12, 2020
sbrannen added a commit that referenced this issue Oct 23, 2020
This commit introduces TestContextAnnotationUtils as a replacement for
MetaAnnotationUtils, with dedicated support for honoring the new
@NestedTestConfiguration annotation and related annotation search
semantics.

MetaAnnotationUtils has been reverted to its previous scope and is now
deprecated.

See gh-19930
sbrannen added a commit that referenced this issue Nov 15, 2020
gh-19930 introduced support for finding class-level test configuration
annotations on enclosing classes when using JUnit Jupiter @nested test
classes, but support for @DynamicPropertySource methods got overlooked
since they are method-level annotations.

This commit addresses this shortcoming by introducing full support for
@NestedTestConfiguration semantics for @DynamicPropertySource methods
on enclosing classes.

Closes gh-26091
sbrannen added a commit to sbrannen/spring-framework that referenced this issue Feb 19, 2022
The TYPE_HIERARCHY_AND_ENCLOSING_CLASSES search strategy for
MergedAnnotations was originally introduced to support @nested test
classes in JUnit Jupiter (see spring-projects#23378).

However, while implementing spring-projects#19930, we determined that the
TYPE_HIERARCHY_AND_ENCLOSING_CLASSES search strategy unfortunately
could not be used since it does not allow the user to control when to
recurse up the enclosing class hierarchy. For example, this search
strategy will automatically search on enclosing classes for static
nested classes as well as for inner classes, when the user probably
only wants one such category of "enclosing class" to be searched.
Consequently, TestContextAnnotationUtils was introduced in the Spring
TestContext Framework to address the shortcomings of the
TYPE_HIERARCHY_AND_ENCLOSING_CLASSES search strategy.

Since this search strategy is unlikely to be useful to general users,
the team has decided to deprecate this search strategy in Spring
Framework 5.3.x and remove it in 6.0.

Closes spring-projectsgh-28079
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: test Issues in the test module type: enhancement A general enhancement
Projects
None yet
Development

No branches or pull requests

7 participants