Remove @MockitoBeanSettings and support for MockitoSession management

[sbrannen]

Overview

We currently have several issues/challenges related to @MockitoBeanSettings, strictness enforcement, and the use of MockitoSession in MockitoTestExecutionListener.

The team needs to decide the fate of @MockitoBeanSettings and MockitoSession management before Spring Framework 6.2 GA.

Issues

I explained some of the challenges in #33690 (comment), and I will list a few additional challenges here.

• @MockitoBeanSettings does not fulfill its original goal: it does not result in custom strictness settings being applied to mocks created via @MockitoBean and @MockitoSpyBean.
• @MockitoBeanSettings currently only applies to mocks created via @Mock/@Spy or those created manually within tests via Mockito.mock(), etc.
• It is impossible for two frameworks to use the MockitoSession simultaneously. Only one MockitoSession can exist at any given time for the current thread.
  • Consequently, attempting to use SpringExtension and the MockitoExtension on the same test class will result either in an exception or in unexpected results (for example, a different strictness applied).
  • Similar problems exist when using Mockito's JUnit 4 integration (Runner and Rules) or in other scenarios where the MockitoSession is used.
• The MockitoTestExecutionListener currently attempts to emulate the behavior of Mockito's own MockitoExtension; however, a Spring TestExecutionListener cannot achieve the same level of integration into JUnit Jupiter's extension model.
  • A Spring TestExecutionListener cannot access the enclosing test instances for @Nested test classes.
  • A Spring TestExecutionListener cannot store state in a parent TestContext (simply because there is no parent).
  • Consequently, MockitoTestExecutionListener actually provides less benefit than using the MockitoExtension when it comes to JUnit Jupiter support.

Example Bugs

The SpringExtension and the MockitoExtension cannot be used in conjunction (since they both attempt to start a new MockitoSession), even though existing projects may already rely on that combination.

For example, Spring Security uses that combination in its test suite. See also commit spring-projects/spring-security@36a408f.

The following fails with an UnfinishedMockingSessionException thrown by org.mockito.junit.jupiter.MockitoExtension.

@ExtendWith(SpringExtension.class)
@ExtendWith(MockitoExtension.class)
class SpringExtensionAndMockitoExtensionTests {

	@Mock
	List<String> mock;

	@Test
	void test() {
	}
}

The following fails with an UnfinishedMockingSessionException thrown by org.springframework.test.context.bean.override.mockito.MockitoTestExecutionListener.

@ExtendWith(MockitoExtension.class)
@ExtendWith(SpringExtension.class)
class MockitoExtensionAndSpringExtensionTests {

	@Mock
	List<String> mock;

	@Test
	void test() {
	}

}

The following should fail with an UnnecessaryStubbingException since the mock created for the @MockitoBean field is stubbed but never used; however, no exception is thrown because the mock created for the @MockitoBean field is created when the ApplicationContext is created and therefore cannot be tracked in the MockitoSession.

@SpringJUnitConfig
@MockitoBeanSettings(STRICT_STUBS)
class MockitoBeanStrictSubsTests {

	@MockitoBean
	List<Integer> mockedList;

	@Test
	void unnecessaryStub() {
		when(mockedList.get(anyInt())).thenReturn(42);
	}

}

Results of exploratory research

MockitoSession is effectively a "Unit of Work" that is bound to the current thread via a ThreadLocal. Mocks created between the "start" and "finish" of the session are tracked for the current thread only. Mocks created outside that session -- for example, before the session or in another thread -- are not tracked by the session. In addition, there is no API that allows one to attach an existing mock to a MockitoSession, and there is no API to query whether a MockitoSession already exists for the current thread.

The following provides an overview of how MockitoSession technically works.

• startMocking() invokes: MockitoAnnotations.openMocks()
• startMocking() registers: org.mockito.internal.junit.UniversalTestListener
• finishMocking() unregisters: org.mockito.internal.junit.UniversalTestListener
• finishMocking() invokes: Mockito.validateMockitoUsage()

Hypothetically (and this is not tested in any way, shape, or form), if we wanted to mimic MockitoSession support for mocks created via @MockitoBean and @MockitoSpyBean, we could track the MockCreationSettings in MockitoBeanOverrideMetadata.createMock() and then register a (subclass of) UniversalTestListener via Mockito.framework().addListener() and invoke CustomSubclassOfUniversalTestListener.onMockCreated(Object, MockCreationSettings) with the mock created by MockitoBeanOverrideMetadata and the saved MockCreationSettings in MockitoTestExecutionListener.beforeTestMethod(), and we could then invoke Mockito.framework().removeListener() and Mockito.validateMockitoUsage() in MockitoTestExecutionListener.afterTestMethod().

However, UniversalTestListener is an internal implementation detail of Mockito and resides in the org.mockito.internal.junit package, and we should ideally not rely on Mockito internals that may change unexpectedly.

Instead, we should approach the Mockito team to discuss alternatives to the current MockitoSession API and semantics that would allow us to provide similar support to mocks that we choose and within a scope that we define.

Related Issues

• Offer a way for MockitoTestExecutionListener to enable strict stubbing #33318
• @MockitoBean reset and MockitoSession management do not work with @Nested tests #33676
• MockitoTestExecutionListener no longer opens mocks in the prepareTestInstance() callback #33690

[simonbasle]

After further discussion, including with the Spring Boot team, we aligned on removing init/close support of Mockito-annotated fields on Framework side. On Boot side this support will be reinstated (using MockitoAnnotations.openMocks) as part of the deprecrated @MockBean features in order to give users an option to maintain the status quo. We will also reach out to the Mockito team to explore possible improvements in Spring Framework 7.