Allow using Cobertura XML report in incremental mode

[The-Compiler]

I'm trying to use diff-cover with mypy to enforce any new code to have type annotations. Here's what I use (via tox):

mypy --cobertura-xml-report {envtmpdir} qutebrowser tests {posargs}
diff-cover --fail-under=100 --compare-branch={env:DIFF_BRANCH:origin/{env:GITHUB_BASE_REF:master}} {envtmpdir}/cobertura.xml

That does seem to work fine - however, due to #6076 (cc @mthuurne), it disables incremental mode (causing the runtime of mypy to go up from <1s to ~15s). Note that in my case I don't care about the XML report only containing changed files, because those are the only ones actually relevant for the annotation coverage diff - thus, I think it would be fine to enable the cache for this use-case.

I understand if this is a too specific case, though - I wonder if it'd make sense to have some "enforce annotations for any new code" functionality in mypy itself (perhaps via the incremental mode)?

[mthuurne]

Disabling the cache entirely when generating reports, like #6076 does, is a rather blunt instrument. I don't know exactly why reports are incomplete when running in incremental mode, but ideally it would be possible to produce a correct coverage report in incremental mode, perhaps by storing more information in the cache.

An alternative would be to let mypy use cached data only for files that are not in the file set it is generating a report for. That way, you could generate a list of changed files and tell mypy to produce a coverage report for only those files.

I do think that mypy should always create a correct report: having an option to force incremental mode when that might lead to an incorrect report is unlikely to be a good long term solution.

[mthuurne]

Actually, passing only the changed files to mypy on the command line may already improve performance. Not sure how much it would help, that depends on how many other modules are pulled in via imports. But it would be worth a try.