Document exit codes for mypy and dmypy

[ilevkivskyi]

See #5982. The rough idea is to exit with:

• 0, if no type errors
• 1, if there are some type errors
• 2, in case of a crash, bad arguments, and other non-standard conditions

[adaszko]

Hi. Should I raise a separate issue for a non-conformity with the above guidelines?

$ pipenv run mypy .
. is not a valid Python package name
$ echo $?
1

I believe that should be a 2.

[gvanrossum]

Thanks -- in general that would be recommended, but in this case it's so simple that I'll just prepare a fix.

[rolandcrosby-check]

Hi, I noticed that adding a Mypy reveal_type call in code that otherwise typechecks correctly causes Mypy to exit with code 1 (it also causes Mypy to not print a "success: no issues found" or "found X errors" message). I can't find any reference to this behavior in Mypy documentation and can't tell if this is expected or if there's a way to work around it.

If Mypy exit codes ever get documented as discussed in this issue, could we ensure that the documentation also addresses the expected impact of reveal_type on the exit code?

[intgr]

1, if there are some type errors
2, in case of a crash, bad arguments, and other non-standard conditions

That's bad. Bugs in mypy plugins can also make it crash with status 1, e.g. typeddjango/django-stubs#760 crashes with 1.

It's the default exit code when Python exits with an unhandled exception, so it's not a good choice for applications to distinguish between "there were type errors" and "mypy crashed"

I was relying on the exit code in the PyCharm plugin, but since it's unreliable, seems like I need to implement some more complicated heuristic.

[meshy]

Bugs in mypy plugins can also make it crash with status 1

I'm seeing the same -- it would be really nice to have an easier way to make the distinction between a crash and errors.

[szabope]

I totally agree: a list of type errors are not errors in mypy, but in the code being checked. In fact, having that output is the expected behavior. So returning with anything other than 0 should indicate that mypy was not able to perform its goal, or at least not completely.
This is the conventional way on POSIX-compatible systems; deviation from the convention will result in surprises and workarounds on the user end.

[meshy]

So returning with anything other than 0 should indicate that mypy was not able to perform its goal

It is certainly useful to be able to make the distinction between "finished with no errors", and "finished with errors" for a linter. It makes scripting and CI integration easier for catching regressions.

For my purposes, I would also like to see a distinction between "finished with errors" and "crashed without finishing".

[szabope]

It is certainly useful to be able to make the distinction between "finished with no errors", and "finished with errors" for a linter. It makes scripting and CI integration easier for catching regressions.

But isn't that distinction is already made by having an output? As listing type errors is the only goal here, empty list means no errors. I think this is the easiest way to use it as an input of some other tool, e.g. in CI integration, and particularly in a shell script.