How to show failed tests in the summary of a finished test run?

[Foxhoundn]

Summary

Right now when the test runner finishes it tells you how many tests & suites failed, but it does not tell you which ones.

It's impossible to figure that out because the test runner spits out thousands of lines of stack trace and HTML dumps.

Is there an option to see which tests failed at the end of the runner?

Test Suites: 4 failed, 29 passed, 33 total
Tests:       9 failed, 168 passed, 177 total // which ones???
Snapshots:   0 total
Time:        551.25 s

CleanShot.2023-12-06.at.14.08.33.mp4

Additional information

No response

Create a reproduction

No response

[chantastic]

is ---verbose the flag you're looking for? https://arc.net/l/quote/wseorsro

[Foxhoundn]

Nope, in the end it still shows just the summary - i'd have to keep watching the run at all times to know which tests have failed (it runs for 12 minutes, I am not gonna do that :) )

Test Suites: 2 failed, 33 passed, 35 total
Tests:       19 failed, 156 passed, 175 total
Snapshots:   0 total
Time:        736.087 s

[Foxhoundn]

Also - please - is there a way to remove the HTML dumps? I Feel like that would help tremendously :(

[yannbf]

Hey there, you can achieve that by setting the environment variable DEBUG_PRINT_LIMIT=<num-of-characters>: https://github.com/storybookjs/test-runner#the-error-output-in-the-cli-is-too-short

e.g. DEBUG_PRINT_LIMIT=0 yarn test-storybook

[Foxhoundn]

Thanks Yann, will try!

[Foxhoundn]

Hi Yann, that did not do anything AFAICS - https://github.com/qoretechnologies/qorus-ide/actions/runs/7991888858/job/21824133970

[Foxhoundn]

Bump please?

[Foxhoundn]

Please please please can someone help with this? I am losing my mind by now, I just want to know why my tests failed.....

CleanShot.2024-04-17.at.15.29.59.mp4

[yannbf]

Hey @Foxhoundn so sorry I missed this for so long. I took a deeper check on your report and the issue is not how lengthy the error message is, but rather how verbose the browser logs are when your test fails! For instance, in this run, a single failure shows the following browser log errors/warnings:

• warn: Warning: React.createFactory() is deprecated and will be removed in a future major release. Consider using JSX or use React.createElement() directly instead.
• error: Warning: Received %s for a non-boolean attribute %s at div
• error: Warning: Received %s for a non-boolean attribute %s at span
• error: Warning: Each child in a list should have a unique "key" prop at div
• error: Warning: Cannot update a component (%s) while rendering a different component (%s). To locate the bad setState() call inside %s, follow the stack trace as described in https://reactjs.org/link/setstate-in-render%s, Unknown, SelectField, SelectField, at SelectField
• error: Warning: Function components cannot be given refs. Attempts to access this ref will fail. Did you mean to use React.forwardRef()?%s%s, Check the render method of ForwardRef., at ReqoreDropdown
• error: Warning: Received %s for a non-boolean attribute %s at span
• error: Warning: Received %s for a non-boolean attribute %s at div
• error: Warning: Received %s for a non-boolean attribute %s at div
• error: Warning: React does not recognize the %s prop on a DOM element. If you intentionally want it to appear in the DOM as a custom attribute, spell it as lowercase %s instead. If you accidentally passed it from a parent component, remove it from the DOM element.%s, asMessage, asmessage, at div
• error: Warning: Each child in a list should have a unique "key" prop.%s%s See https://reactjs.org/link/warning-keys for more information.%s, Check the render method of AutoField. at SubField

So what you can do is to change the logLevel configuration to none and you should not get so much noise anymore (though it might be nice to fix the errors displayed in the browser):
https://github.com/storybookjs/test-runner#loglevel

I hope this helps! If it doesn't, I'll make it my mission to fix your frustration!

[Foxhoundn]

Hey @Foxhoundn so sorry I missed this for so long. I took a deeper check on your report and the issue is not how lengthy the error message is, but rather how verbose the browser logs are when your test fails! For instance, in this run, a single failure shows the following browser log errors/warnings:

• warn: Warning: React.createFactory() is deprecated and will be removed in a future major release. Consider using JSX or use React.createElement() directly instead.

• error: Warning: Received %s for a non-boolean attribute %s at div

• error: Warning: Received %s for a non-boolean attribute %s at span

• error: Warning: Each child in a list should have a unique "key" prop at div

• error: Warning: Cannot update a component (%s) while rendering a different component (%s). To locate the bad setState() call inside %s, follow the stack trace as described in https://reactjs.org/link/setstate-in-render%s, Unknown, SelectField, SelectField, at SelectField

• error: Warning: Function components cannot be given refs. Attempts to access this ref will fail. Did you mean to use React.forwardRef()?%s%s, Check the render method of ForwardRef., at ReqoreDropdown

• error: Warning: Received %s for a non-boolean attribute %s at span

• error: Warning: Received %s for a non-boolean attribute %s at div

• error: Warning: Received %s for a non-boolean attribute %s at div

• error: Warning: React does not recognize the %s prop on a DOM element. If you intentionally want it to appear in the DOM as a custom attribute, spell it as lowercase %s instead. If you accidentally passed it from a parent component, remove it from the DOM element.%s, asMessage, asmessage, at div

• error: Warning: Each child in a list should have a unique "key" prop.%s%s See https://reactjs.org/link/warning-keys for more information.%s, Check the render method of AutoField. at SubField

So what you can do is to change the logLevel configuration to none and you should not get so much noise anymore (though it might be nice to fix the errors displayed in the browser):

https://github.com/storybookjs/test-runner#loglevel

I hope this helps! If it doesn't, I'll make it my mission to fix your frustration!

I will try to set the log level to none - but the HTML dumps is what is taking the most space :(

[yannbf]

The HTML dumps should be truncated correctly in the test error message. You get duplicated, untruncated ones from the browser logs. I think that's the issue!

[nickichi]

@Foxhoundn
Hello. I've also been investigating ways to eliminate meaningless HTML output. You may have already noticed, but with the recent addition of errorMessageFormatter, my issue has been resolved. Here's the code I'm using:

// .storybook/test-runner.ts
// @storybook/test-runner ver 0.19.1
import type { TestRunnerConfig } from '@storybook/test-runner'

const config: TestRunnerConfig = {
  errorMessageFormatter: message => {
    const msg = message.match(/Message:([\s\S]*?)Ignored nodes:/)
    return msg ? msg[1] : message
  }
}
export default config

I hope this helps.