react-scripts test (or Jest) is reporting exit code 0 on test failures

[thedanchez]

Is this a bug report?

Possibly

Did you try recovering your dependencies?

Yes

Environment

System:
    OS: macOS High Sierra 10.13.6
    CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  Binaries:
    Node: 8.12.0 - ~/.nvm/versions/node/v8.12.0/bin/node
    Yarn: 1.10.1 - /usr/local/bin/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v8.12.0/bin/npm
  Browsers:
    Chrome: 75.0.3770.80
    Firefox: 66.0.2
    Safari: 12.1
  npmPackages:
    react: ^16.6.3 => 16.6.3
    react-dom: ^16.6.3 => 16.6.3
    react-scripts: 2.1.8 => 2.1.8
  npmGlobalPackages:
    create-react-app: Not Found

Steps to Reproduce

1. Create a CRA project with tests
2. Create a bash script that encapsulates the react-scripts test command execution
3. Output the exit code result from within that wrapping bash script

Expected Behavior

I expect the exit code to be non-zero when tests fail

Actual Behavior

react-scripts test is reporting a 0 exit code

Reproducible Demo

None yet

[bugzpodder]

You can try setting CI=true to get a non-zero exit code.

[heyimalex]

Duplicate of #7026 I think. Last time I checked we're doing everything right, but jest doesn't return the appropriate error code.

[thedanchez]

Setting the CI env variable works but the usage of that for local test runs doesn't seem right to me. We also lose the pretty output.

Been digging around for the past couple of hours and found that adding --bail in the CLI works as well.

[goranefbl]

I think its the same problem as #6944
I have a Husky configured and without CI env, even though tests are failing, Husky is not stopping commits from going through. Adding of CI works, but as @thedanchez pointed out, we are losing pretty output.

Update:
Seems that --bail keeps the pretty output.

[silverwind]

Confirmed, steps to reproduce:

$ npx create-react-app exittest
$ cd exittest
$ echo fail > src/App.js
$ npx react-scripts test --watchAll=false
$ echo $?
0

[bugzpodder]

I think add --bail works as others pointed out

[silverwind]

Imho, a proper exit code should be set by default and not only with obscure arguments like --bail.

[bugzpodder]

Agreed, PRs welcome!

[mAAdhaTTah]

Also --bail will stop running tests on the first error, rather than running the full test suite then erroring.