`MockitoTestExecutionListener` no longer opens mocks in the `prepareTestInstance()` callback

[wilkinsona]

Affects: 6.2.0-SNAPSHOT

This is a recent regression from 6.2.0-RC1. I believe it was introduced in eb4bf1c#diff-02b9cc5ba2d6121e1a5f63ad339bb82fa8f3531d568c43d07fac61ade962c0cd where the override of prepareTestInstance was removed. The result is that trying to call a @Mock-annotated field in a @BeforeAll method fails with an NPE.

This should reproduce the regression:

package com.example;

import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.BDDMockito.given;

import java.util.List;

import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.springframework.test.context.junit.jupiter.SpringExtension;

@ExtendWith(SpringExtension.class)
class MockInitInBeforeAllTests {
	
	@Mock
	private static List<String> mock;

	@BeforeAll
	static void setUp() {
		given(mock.size()).willReturn(1);
	}

	@Test
	void shouldSetUpSuccessfully() {
		assertThat(mock.size()).isEqualTo(1);
	}

}

Another, more advanced variant that uses a custom test instance lifecycle and which is closer to the Spring Boot test that found the regression also fails with 6.2.0-SNAPSHOT:

package com.example;

import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.BDDMockito.given;

import java.util.List;

import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.TestInstance.Lifecycle;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.springframework.test.context.junit.jupiter.SpringExtension;

@TestInstance(Lifecycle.PER_CLASS)
@ExtendWith(SpringExtension.class)
class MockInitInBeforeAllTests {
	
	@Mock
	private List<String> mock;

	@BeforeAll
	void setUp() {
		given(this.mock.size()).willReturn(1);
	}

	@Test
	void shouldSetUpSuccessfully() {
		assertThat(mock.size()).isEqualTo(1);
	}

}

[sbrannen]

Thanks for reporting the issue, @wilkinsona. 👍🏻

The result is that trying to call a @Mock-annotated field in a @BeforeAll method fails with an NPE.

I confirm that this is a change in behavior in Spring Framework since 6.2 RC1, and it is in fact caused by the removal of the prepareTestInstance() override in MockitoTestExecutionListener.

However, that change was made in order to provide better support for @Nested test classes with the MockitoTestExecutionListener.

If we reinstate the prepareTestInstance() override as it was, we end up closing the MockitoSession for the enclosing class when we open a new MockitoSession for the @Nested class, and that means that mocks created in the enclosing class will not be tracked by the MockitoSession created for the @Nested class: in other words, strictness and unnecessary stubbing would not be supported for mocks created in the enclosing class.

On the other hand, if we reintroduce the prepareTestInstance() override and do not close the existing MockitoSession, we get an UnfinishedMockingSessionException for the @⁠Nested test class due to the fact that a MockitoSession was already created for the current thread for the enclosing test class.

In retrospect, neither of those scenarios is desirable, and I'll address that topic in a new GitHub issue.

In addition, I confirmed that Spring Boot never supported the use cases you have provided, and Mockito's own MockitoExtension also does not support initialization for @Mock on a static field before a @BeforeAll method is invoked.

For example, the following all fail due to a NullPointerException.

@SpringBootTest
@TestExecutionListeners({
	org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener.class,
	org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener.class
})
class MockInitInBeforeAllWithSpringBootTests {

	@Mock
	static List<String> mock;

	@BeforeAll
	static void setUp() {
		given(mock.size()).willReturn(1);
	}

	@Test
	void shouldSetUpSuccessfully() {
		assertThat(mock.size()).isEqualTo(1);
	}

}

@SpringBootTest
@TestExecutionListeners({
	org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener.class,
	org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener.class
})
class MockInitInBeforeEachWithSpringBootTests {

	@Mock
	List<String> mock;

	@BeforeEach
	void setUp() {
		given(mock.size()).willReturn(1);
	}

	@Test
	void shouldSetUpSuccessfully() {
		assertThat(mock.size()).isEqualTo(1);
	}

}

@ExtendWith(MockitoExtension.class)
class MockInitInBeforeAllWithMockitoExtensionTests {

	@Mock
	private static List<String> mock;

	@BeforeAll
	static void setUp() {
		given(mock.size()).willReturn(1);
	}

	@Test
	void shouldSetUpSuccessfully() {
		assertThat(mock.size()).isEqualTo(1);
	}

}

However, the MockitoExtension and Spring Framework 6.2 snapshots do support initialization for @Mock on an instance field before a @BeforeEach method is invoked, as demonstrated in the following examples which succeed.

@ExtendWith(MockitoExtension.class)
class MockitoInitInBeforeEachWithMockitoExtensionTests {

	@Mock
	private List<String> mock;

	@BeforeEach
	void setUp() {
		given(mock.size()).willReturn(1);
	}

	@Test
	void shouldSetUpSuccessfully() {
		assertThat(mock.size()).isEqualTo(1);
	}

}

@ExtendWith(SpringExtension.class)
class MockitoInitInBeforeEachWithSpringExtensionTests {

	@Mock
	private List<String> mock;

	@BeforeEach
	void setUp() {
		given(mock.size()).willReturn(1);
	}

	@Test
	void shouldSetUpSuccessfully() {
		assertThat(mock.size()).isEqualTo(1);
	}

}

In summary, if we wish to continue using the MockitoSession I do not think we can reliably support the inialization of static @Mock, @Spy or @Captor fields before a @BeforeAll method is invoked, since we need to properly support @Nested test classes as well.

In light of that, I am closing this issue.

[sbrannen]

• Superseded by Remove @MockitoBeanSettings and support for MockitoSession management #33692

[wilkinsona]

I’m surprised that this has been closed, particularly with the assertion that “Spring Boot never supported the use cases you have provided” when, as I mentioned above, there is a test in Spring Boot that started failing with 6.2.0-SNAPSHOT. It succeeds with 6.1.x (Boot 3.3) and with 6.2.0-RC1. You can find the test class here. It’s ConfigureMockInBeforeAll that is affected by the regression.

[sbrannen]

Hi Andy,

Thanks for the feedback.

I’m surprised that this has been closed, particularly with the assertion that “Spring Boot never supported the use cases you have provided” when, as I mentioned above, there is a test in Spring Boot that started failing with 6.2.0-SNAPSHOT. It succeeds with 6.1.x (Boot 3.3) and with 6.2.0-RC1.

I misunderstood your original claim. I wrongly assumed you meant the regression was for behavior introduced in Spring Framework that did not exist previously in Spring Boot, but now I see that the MockitoTestExecutionListener used to open mocks in the prepareTestInstance() callback here.

My apologies!

I'm reopening this issue to make sure we take that into consideration.