Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nullpointer Exception when using JUnit 5 @Nested tests #490

Closed
slu-it opened this issue Mar 14, 2018 · 6 comments
Closed

Nullpointer Exception when using JUnit 5 @Nested tests #490

slu-it opened this issue Mar 14, 2018 · 6 comments

Comments

@slu-it
Copy link

slu-it commented Mar 14, 2018

Hello Spring REST Docs Team!

It seems that JUnit 5's @Nested tests don't work with auto-configured Spring REST Docs.

Executing the following simple test class will result in a NullPointerException:

package example;

import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.restdocs.AutoConfigureRestDocs;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import org.springframework.test.web.servlet.MockMvc;

import static org.hamcrest.CoreMatchers.equalTo;
import static org.springframework.restdocs.mockmvc.MockMvcRestDocumentation.document;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@SpringBootTest
@AutoConfigureMockMvc
@AutoConfigureRestDocs
@ExtendWith(SpringExtension.class)
class FailingTest {

    @Autowired
    MockMvc mockMvc;

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                    .andExpect(status().is2xxSuccessful())
                    .andExpect(jsonPath("message", equalTo("Hello World!")))
                    .andDo(document("hello-get-200"));
        }

    }

}

The resulting stacktrace:

java.lang.NullPointerException
	at org.springframework.restdocs.ManualRestDocumentation.beforeOperation(ManualRestDocumentation.java:89)
	at org.springframework.restdocs.mockmvc.MockMvcRestDocumentationConfigurer$ConfigurerApplyingRequestPostProcessor.postProcessRequest(MockMvcRestDocumentationConfigurer.java:100)
	at org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder.postProcessRequest(MockHttpServletRequestBuilder.java:755)
	at org.springframework.test.web.servlet.MockMvc.perform(MockMvc.java:154)
	at example.FailingTest$NestedTests.testAndDocumentation(FailingTest.java:33)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:436)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:170)
	at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:166)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:113)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:58)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:112)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:55)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:43)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:74)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

Tested with Spring Boot 2.0.1.BUILD-SNAPSHOT, Spring REST Docs 2.0.0.RELEASE and JUnit 5.1.0 on 2018-03-14 at around 11:00 PM.

I created a simple example application (thx start.spring.io ;) ) to reproduce the issue:
https://github.com/slu-it/bug-spring-restdocs-nested-tests

Note: This might be related to the following issue I opened in Spring Boot:
spring-projects/spring-boot#12470

@wilkinsona
Copy link
Member

wilkinsona commented Mar 15, 2018

Thanks for the sample. The NullPointerException is occurring because your inner test class isn't a Spring Boot test. I see you've corrected that in AlsoFailingTest that fails for a different reason. It's failing due to what is either a bug in or a limitation of the testing support provided by Spring Framework. I've opened https://jira.spring.io/browse/SPR-16595.

Note that these problems are specific to @AutoConfigureMockMvc and @AutoConfigureRestDocs. If you configure MockMvc and REST Docs without relying on Boot's test auto-configuration, this should work.

@lzhoucs
Copy link

lzhoucs commented Mar 17, 2019

Just want to check in on the status of this issue. It seems the limitation mentioned above with junit 5 @Nested class is fixed externally in spring framework 5.1.0, according to spring-projects/spring-framework#21136 (comment), however I am still running into the same NullPointerException in my @Nested class when I use @AutoConfigureMockMvc and @AutoConfigureRestDocs.

We have a lot of tests in Nested class, and they work fine when restdocs logic is removed. So it seems the problem is specific to restdocs. Converting those tests into top level class ones is an option, but not ideal if there's other way around.

Manually configure mockMvc with rest docs might not be an easy option, as there's a large gap to be filled, as mentioned #41 (comment). We do have spring security filters that is not included by default when going manual.

@wilkinsona
Copy link
Member

@lzhoucs AFAIK, this should work. If you believe that's not the case, please take the time to provide a minimal sample as @slu-it did.

@lzhoucs
Copy link

lzhoucs commented Mar 19, 2019

Thanks @wilkinsona for the response. I thought it might still be a known issue so I just tried to confirm. Knowing it isn't, I just created a quick demo branch: https://github.com/lzhoucs/spring-restdocs/tree/junit5-nested-issue-demo where I modified the SampleJUnit5ApplicationTests as follows:

@SpringBootTest
@AutoConfigureMockMvc
@AutoConfigureRestDocs
public class SampleJUnit5ApplicationTests {

	@Autowired
	private MockMvc mockMvc;

	@Test
	public void sample() throws Exception {
		mockMvc.perform(get("/hello"))
				.andExpect(status().isOk())
				.andDo(document("sample"));
	}

	@Nested
	class NestedTests {
		@Test
		public void sampleNested() throws Exception {
			mockMvc.perform(get("/hello"))
				.andExpect(status().isOk())
				.andDo(document("sample"));
		}
	}
}

sampleNested test is failing right now.

@wilkinsona
Copy link
Member

The configuration you've tried doesn't match the recommendations above.

The NullPointerException is occurring because your inner test class isn't a Spring Boot test

With the configuration that you've shared, NestedTests creates a new application context so the bean that should set REST Docs up correctly isn't found.

You can avoid the problem by configuring your nested class like this:

@Nested
@SpringBootTest
@AutoConfigureMockMvc
@AutoConfigureRestDocs
class NestedTests {

	@Autowired
	MockMvc mockMvc;

	@Test
	public void sampleNested() throws Exception {
		this.mockMvc.perform(get("/"))
			.andExpect(status().isOk())
			.andDo(document("sample"));
	}

}

You need to inject MockMvc again as, despite the apparently identical configuration, NestedTests still creates a second application context and you need the MockMvc instance from that context not the context of SampleJUnit5ApplicationTests.

This doesn't feel right to me, but it's out of the control of Spring REST Docs. @sbrannen how are nested tests expected to work in Framework 5.1? What configuration is required to have a nested test reuse the context, test execution listeners, etc from its outer class?

@sbrannen
Copy link
Member

This doesn't feel right to me, but it's out of the control of Spring REST Docs. @sbrannen how are nested tests expected to work in Framework 5.1? What configuration is required to have a nested test reuse the context, test execution listeners, etc from its outer class?

The behavior is 5.1 is the same as in 5.0.

The current goal is to address this in 5.2: spring-projects/spring-framework#19930

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants