`CleanupMode.ON_SUCCESS` for class-level `@TempDir` does not work for failure in `@Test` method

[delanym]

As discussed in #2159 setting @TempDir(cleanup = CleanupMode.ON_SUCCESS) does not retain the directory after a test failure, while @TempDir(cleanup = CleanupMode.NEVER) retains the directory, and @TempDir(cleanup = CleanupMode.ALWAYS) removes the directory.

This time I'm using a different plugin.

        <plugin>
          <groupId>de.sormuras.junit</groupId>
          <artifactId>junit-platform-maven-plugin</artifactId>
          <extensions>true</extensions>
          <version>1.1.7</version>
          <configuration>
            <isolation>ABSOLUTE</isolation>
            <executor>JAVA</executor>
            <javaOptions>
              <inheritIO>true</inheritIO>
            </javaOptions>
          </configuration>
        </plugin>

JUnit 5.10.0
Maven 3.9.5

[delanym]

I see this in the console output

WARNING: Delegated to the 'execute' command.
         This behaviour has been deprecated and will be removed in a future release.
         Please use the 'execute' command directly.

Perhaps extensionContext.getExecutionException().isPresent() is always false?
https://github.com/junit-team/junit5/blob/766561f5884600ee91c1845c647c8ce21c7c38da/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TempDirectory.java#L285C6-L285C6

[sbrannen]

Hi @delanym,

Thanks for raising the issue.

If you want us to investigate this, please provide a minimal sample application that we can run ourselves (preferably a public Git repository or a ZIP file attached to this issue).

[delanym]

class BasicTest {

  @TempDir(cleanup = CleanupMode.ON_SUCCESS)
  static Path tempDir;
  Path temp = BasicTest.tempDir.resolve("");

  @Test
  public void test() {
    System.out.println(temp);
  }
}

[sbrannen]

There is no test failure in the above example.

When I run it, it prints:

/var/folders/_7/ll0c1sh52c5dfrvmq7j8l1x00000gn/T/junit13377456877063442804

And when I execute open /var/folders/_7/ll0c1sh52c5dfrvmq7j8l1x00000gn/T/junit13377456877063442804, I get the following:

The file /var/folders/_7/ll0c1sh52c5dfrvmq7j8l1x00000gn/T/junit13377456877063442804 does not exist.

Which indicates that the folder was deleted after successful completion of the test class.

[sbrannen]

However, if I use the following modified version of the test class you provided...

class BasicTest {

	@TempDir(cleanup = CleanupMode.ON_SUCCESS)
	Path tempDir;


	@Test
	public void test() {
		Path temp = tempDir.resolve("");
		System.out.println(temp);
		fail("Boom!");
	}

}

The temp folder is then retained after the test failure.

If you change the tempDir field to static, the temp folder is not retained.

So perhaps that is what you are raising an issue about -- yes?

[delanym]

Right

[sbrannen]

Perhaps extensionContext.getExecutionException().isPresent() is always false?

Now that we have verified your actual use case, I can answer this question.

Yes, I believe that's the case.

The team will need to investigate this to be certain, but I believe the following is the cause of this behavior.

The ExtensionContext for a class-level temp-dir (static field) will not return an execution exception thrown by a test method within that class. Consequently, the CloseablePath (created the for static temp-dir field) does not know that it should retain the temp-dir after a test method failure.