[Request] Hide the JUnit Callstack from the stacktrace

[leogott]

There are (few, but there are) cases in which I'd like to take a look at the callstack for a failed (as opposed errored) test. Mainly when using assertThrows. However I would like to have it shortened by default so that in the simplest case every line starting with at org.junit is replaced with the ... X more lines just like the end is shortened right now.

I'd be satisfied with a setting like "shorten lines matching this regex from the stack trace".

For example when assertThrows fails, because instead of ExpectedException I got DifferentException, I'd love the output to look like

org.opentest4j.AssertionFailedError: Unexpected exception type thrown ==> expected: <org.example.ExpectedException> but was: <org.example.DifferentException>
    at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:65)
    ... 70 more lines
Caused by: org.example.DifferentException: Message
    at x
    at y
    at z

[jdneo]

@leogott Could you give an example of what's the original stack trace looks like?

In the example you gave, why we just hide the first part of the stack trace but leave the second one as it is?

[leogott]

For a small example I wrote (junit5 assert throws expects A but receives B caused by C), I get this unabbreviated result.

Every line beginning with at org.junit.** (for example) is part of the testing implementation and I don't care about the junit internals, because I'm confident my own code is the likely cause of the failed test.

I don't care about the callstack of the AssertionFailedError (beyond that it was in line 25 of AppTest.java) , because it was thrown by the assertThrows method.

I do however care about the Cause (the exception of wrong type), and where it originated.

So in this case I'm getting the following information:

%TESTC  1 v2
%TSTTREE2,org.example.AppTest,true,1,false,1,AppTest,,[engine:junit-jupiter]/[class:org.example.AppTest]
%TSTTREE3,someFeature_stackTrace(org.example.AppTest),false,1,false,2,someFeature_stackTrace(),,[engine:junit-jupiter]/[class:org.example.AppTest]/[method:someFeature_stackTrace()]
%TESTS  3,someFeature_stackTrace(org.example.AppTest)
%FAILED 3,someFeature_stackTrace(org.example.AppTest)
%TRACES 
org.opentest4j.AssertionFailedError: Unexpected exception type thrown ==> expected: <java.io.IOException> but was: <java.lang.IllegalArgumentException>
        at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:65)
        at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:37)
        at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3007)
        at org.example.AppTest.someFeature_stackTrace(AppTest.java:25)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
        at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
        at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
        at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
        at java.util.ArrayList.forEach(ArrayList.java:1259)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
        at java.util.ArrayList.forEach(ArrayList.java:1259)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:84)
        at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:98)
        at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Caused by: java.lang.IllegalArgumentException: Argument must not be null
        at org.example.App.someFeatureImTesting(App.java:20)
        at org.example.AppTest.lambda$0(AppTest.java:26)
        at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:55)
        ... 70 more
Caused by: java.lang.NullPointerException
        at org.example.App.someFeatureImTesting(App.java:17)
        ... 72 more
%TRACEE 
%TESTE  3,someFeature_stackTrace(org.example.AppTest)
%RUNTIME175


> Test run finished at 21.8.2021, 01:23:53 <

But this is what I care about:

TEST: someFeature_stackTrace(org.example.AppTest)
RESULT: failed
TRACE:
org.opentest4j.AssertionFailedError: Unexpected exception type thrown ==> expected: <java.io.IOException> but was: <java.lang.IllegalArgumentException>
        ... 3 more
        at org.example.AppTest.someFeature_stackTrace(AppTest.java:25)
        ... X more
Caused by: java.lang.IllegalArgumentException: Argument must not be null
        at org.example.App.someFeatureImTesting(App.java:20)
        at org.example.AppTest.lambda$0(AppTest.java:26)
        ... 71 more
Caused by: java.lang.NullPointerException
        at org.example.App.someFeatureImTesting(App.java:17)
        ... 72 more

Would it be possible to at least cut the callstack for the AssertionFailed Exception that fails the Test?

[jdneo]

I see, so that is to only reserve the stack trace which belongs to the source?

[leogott]

Oh you're correct. I didn't look at it that way, but it absolutely is just the parts of the callstack within the source at test.

Yes, I'd like to see a filtered view or something that cuts away the callstack that doesn't belong to the source.

[jdneo]

Get a very similar feedback from a customer interview. He hope the stacktrace can be hidden by clicking some buttons like show more, show less

[jdneo]

Hi @leogott,

I pushed a change to filter out those 'internal' traces. Currently we don't have a setting to control it. But we can definitely add a setting for that if users require.

You can try the new behavior in the next pre-release of the extension (should be available tomorrow).

Let me know if you have any feedback, thanks!

[jdneo]

To verify this issue

• Run a JUnit test case which will fail.
• Check the stacktrace, it should not contain following packages/classes:
  • org.eclipse.jdt.internal.junit.runner.*
  • org.eclipse.jdt.internal.junit4.runner.*
  • org.eclipse.jdt.internal.junit5.runner.*
  • org.eclipse.jdt.internal.junit.ui.*
  • junit.framework.TestCase
  • junit.framework.TestResult
  • junit.framework.TestResult$1
  • junit.framework.TestSuite
  • junit.framework.Assert
  • org.junit.*
  • java.lang.reflect.Method.invoke
  • sun.reflect.*
  • jdk.internal.reflect.*

// cc @testforstephen