Description
If mypy daemon generates errors in multiple files, successive daemon runs can change the order of messages even if they are otherwise identical (e.g. the first run produces messages from mod1 first followed by messages from mod2, and the second runs produces errors from mod2 first and mod1 next). Try to keep the order of error messages consistent across runs, since the varying order can be distracting when you are trying to fix errors one by one.
Suggested approach:
- Store the order of files where we reported errors after each daemon run as a list
- If we had errors in the previous run, sort the errors in the next run so that the order of files matches the previous run (and errors in additional files are at the end, for example)
Errors are already sorted consistently within a single file, but not across multiple files.
Another way would be to sort the messages by the file before reporting them, but this would be a bigger change conceptually and might have some issues (e.g. can't stream errors during the initial run). Preserving the order of errors from the immediate previous run shouldn't cause any additional issues. Related issue: #2144
Test cases to cover:
- Initial, non-daemon run generates errors.
- Two successive incremental runs generate errors.
- There is a successful run between runs that generate errors. In this case the errors can be reported in an arbitrary order.