Clarifying documentation on including a top-level @TestConfiguration class in a test

[kesslerj]

A test configuration annotated with @TestConfiguration does not override a bean, if it is defined as a separate class. Only using @TesConfiguration on an inner class works in this case.

Not working:

@TestConfiguration
public class ExternalTestConfig {
	@Bean
	public MyBean myBean() {
		return Mockito.mock(MyBean.class);
	}	
}

@SpringBootTest(properties = "spring.main.allow-bean-definition-overriding=true")
@Import(ExternalTestConfig.class)
public class ExternalTestConfigApplicationTest {
	@Autowired
	private MyBean myBeanMock;

	@Test
	void contextLoads() {
		// test fails, if myBeanMock is not a mock
		Mockito.verifyNoInteractions(myBeanMock);
	}
}

Working:

@SpringBootTest(properties = "spring.main.allow-bean-definition-overriding=true")
class InnerClassTestConfigApplicationTest {
	@Autowired
	private MyBean myBeanMock;

	@Test
	void contextLoads() {
		// test fails, if myBeanMock is not a mock
		Mockito.verifyNoInteractions(myBeanMock);
	}
	
	@TestConfiguration
	public static class TestConfig{
		
		@Bean
		public MyBean myBean() {
			return Mockito.mock(MyBean.class);
		}
	}
}

My example uses spring boot 2.6.6, code can be found here: https://github.com/spring-boot-demos/demo-testing-bean-overriding

I read several blogs and articles and found some other examples, which do not work, e.g. this blog with example code which is very similar to my example code.

In my understanding, the behaviour of a class annotated with @TestConfiguration should be the same, no matter if defined as inner or separate class.

[mhalbritter]

Thanks for providing the reproducer project!

Our documentation doesn't mention that @TestConfiguration inner classes override the configuration of the application at all:

a nested @TestConfiguration class is used in addition to your application’s primary configuration.

I agree that it's strange that the behavior changes when using @TestConfiguration in nested mode vs. in imported mode. As a workaround, you can annotate the @Bean methods in the testconfiguration as @Primary, which overrides the bean definition from your application.

I'll flag this as team attention, let's see what the rest of the team thinks.

[wilkinsona]

As far as I know, Spring Framework's testing support doesn't process @Import on a test class so ExternalTestConfig is being ignored in the non-working example.

Instead of @Import you can use the classes attribute of @SpringBootTest or the separate @ContextConfiguration annotation:

@SpringBootTest(properties = "spring.main.allow-bean-definition-overriding=true")
@ContextConfiguration(classes = ExternalTestConfig.class)
public class ExternalTestConfigApplicationTest {

@SpringBootTest(properties = "spring.main.allow-bean-definition-overriding=true", classes = ExternalTestConfig.class)
public class ExternalTestConfigApplicationTest {

I think we need to correct the documentation as this section shows @Import being used on a test class in a way that does not work. While we are there, I think it would be worth clarifying the following as well:

When placed on a top-level class, @TestConfiguration indicates that classes in src/test/java should not be picked up by scanning.

To me, this is saying that a single top-level class annotated with @TestConfiguration will prevent any classes in src/test/java from being picked up by component scanning. I don't believe that's the case. It's only classes annotated with @TestConfiguration that will not be picked up. Any regular @Configuration classes will still be found.

[mhalbritter]

Yep, you're right Andy. My "workaround" with the @Primary doesn't work. @Import doesn't work on test classes.

[kesslerj]

@wilkinsona @mhalbritter Thank you for your research, but I can not agree. @Import works fine on test classes. The only thing that does not work if using @Import on test classes, is to override a bean configuration, which was defined using @Bean. Overriding a bean definition, which is defined by using e.g. @Service, or even creating any other bean, works fine.

I updated my example code to proove this.

[wilkinsona]

Apologies, @kesslerj. I'd forgotten about Boot's ImportsContextCustomizer that allows @Import to be used on a test class. It was added primarily in support of @ImportAutoConfiguration and isn't really intended for your usage.

The problem you're seeing is due to ordering and the different times at which beans are defined between component scanning, @Import, and nested @TestConfiguration. @Import processing intentionally occurs early in the creation of the application context. This results in the @Imported ExternalTestConfig being processed before de.aclue.beansdemo.BeansDemoApplication. As a result, myBean on ExternalTestConfig is defined first and is then overridden by myBean on de.aclue.beansdemo.BeansDemoApplication. The ordering is not the same when component scanning is involved and in that case, the @Service-annotated AnotherBean is defined before anotherBean on ExternalTestConfig. As a result, anotherBean on ExternalTestConfig overrides the @Service-annotated AnotherBean.

As I said above, you can use @ContextConfiguration or the classes attribute on @SpringBootTest to get things working. With this change in place, all of the tests in your sample project pass. We'll use this issue to update the documentation to steer people in this direction.

[kesslerj]

@wilkinsona thank you for your detailed answer. Unfortunately none of the solutions is really satisfying our needs.

Let me quickly explain our desired setup:
We are always working with an abstract test class which is then extended by (almost) all other integration test classes. This way we are able to work with (most times) only one spring context which is started and then used for all tests. Our approach is to mock as less beans as possible and reuse as much of the original app configuration as possible.

Our previous approach was to use @Import along with an @TestConfiguration in a separate class. As it turned out in this issue, this solutions fails in overriding beans defined in a @Bean annotated method.

I only see two other approaches:

1. @TestConfiguration as an inner class
   Unfortunately this approach does not work for an abstract test class. As we don't want to go back to monolithic test classes with a config and a real bunch of test methods - or copy the @TestConfiguration annotated inner class in all our test classes - this can not be the solution.
2. @ContextConfiguration on our abstract test class, enumerating all @Configuration classes from our application code, followed by our test configuration class.
   This is really fragile solution, as we have to declare all of our configurations manually, which is likely to be forgotten for a new implemented configuration. We have to redefine our complete configuration instead of just overwrite some beans of it.

I extended my example repository and added a setup with an abstract test class, see branch https://github.com/spring-boot-demos/demo-testing-bean-overriding/tree/abstract-test-class.

Do you have any other ideas how we can go on with our abstract test class in a comfortable way? Is it somehow possible to not redefine all used configurations with @ContextConfiguration, but only add one (at the end)?