mypy 0.982 run 15 minutes (previous version needs only one minute)

[Behoston]

Bug Report

My project is not open-source but I can give you (mypy maintainers) access to repository on gitlab.

After updating mypy to newest version it takes 15 minutes to complete on CI and 8 minutes on my PC. When I rollback to previous release it will works fine (about 15 seconds on local machine and about 45 on CI).

I have base class for about 100 subclasses and mypy hangs always in same place.

LOG:  Deleting dostanesie.core.formulas.university.ur.base dostanesie/core/formulas/university/ur/base.py dostanesie/core/formulas/university/ur/base.meta.json dostanesie/core/formulas/university/ur/base.data.json
LOG:  Processing SCC singleton (dostanesie.core.formulas.university.uph.base) as inherently stale with stale deps (dostanesie.core.formulas.code)
LOG:  Deleting dostanesie.core.formulas.university.uph.base dostanesie/core/formulas/university/uph/base.py dostanesie/core/formulas/university/uph/base.meta.json dostanesie/core/formulas/university/uph/base.data.json
LOG:  Processing SCC singleton (dostanesie.core.formulas.university.up_wroclaw.base) as inherently stale with stale deps (dostanesie.core.formulas.code)
LOG:  Deleting dostanesie.core.formulas.university.up_wroclaw.base dostanesie/core/formulas/university/up_wroclaw/base.py dostanesie/core/formulas/university/up_wroclaw/base.meta.json dostanesie/core/formulas/university/up_wroclaw/base.data.json
LOG:  Processing SCC singleton (dostanesie.core.formulas.university.up_poznan.base) as inherently stale with stale deps (dostanesie.core.formulas.code)
LOG:  Deleting dostanesie.core.formulas.university.up_poznan.base dostanesie/core/formulas/university/up_poznan/base.py dostanesie/core/formulas/university/up_poznan/base.meta.json dostanesie/core/formulas/university/up_poznan/base.data.json
LOG:  Processing SCC singleton (dostanesie.core.formulas.university.up_lublin.base) as inherently stale with stale deps (dostanesie.core.formulas.code)

I tried deleting mypy cache dir locally, on CI we build new docker image each run so there is no mypy cache.

To Reproduce

I really don't know :(

Expected Behavior

Mypy should run about one minute for version 0.971.

LOG:  Build finished in 14.859 seconds with 276 modules, and 0 errors

Actual Behavior

On CI it takes more than 15 minutes, on local machine 8 minutes.

Your Environment

• Mypy version used: 0.982
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files):

[mypy]
ignore_missing_imports = true

• Python version used: Python 3.10.7+

Full mypy log

[ilevkivskyi]

Such things are hard to diagnose. You can try bisecting it to a specific mypy commit, then it will be much easier.

[Behoston]

I definitely will try today. Also I will try to extract problematic part of our code and share it on GH

[Behoston]

3ae19a2 this takes more than 25 minutes

I run this in the background and killed after 25 minutes, it takes longer than previous because I use my PC.

[JukkaL]

I looked at the commit, and a few things seemed potentially suspicious: changes in sum(), subprocess and unittest. If the file where mypy is very slow uses some of these, could you try commenting out the relevant lines to see if it speeds things up?

[Behoston]

replaced almost all sum with any and it takes 30 seconds for commit above (not compiled) and 14 seconds for newest version (compiled). We have something between 100 and 200 sum in this package.

[JukkaL]

That would indicate that the sum change is likely the root cause for the regression, since when using sum the run was was over 25 minutes. Does this sound right?

[Behoston]

In my case: True

we can stick to 0.971, newest version of mypy is not crucial. However, it can affect many other projects CI/CD, especially calculation-dependent (our backend is one big calculator with hundreds calculator classes and some REST API)

[JukkaL]

Yeah, this seems pretty bad and we should try to fix the regression.

It would be very helpful if you could you try to narrow this down a bit -- it's possible that there is some specific call or calls to sum that are very slow. For example, replace the first 50% of the sum calls and then the remaining 50%, in order to narrow things down/bisect (and measure afterwards), and continue until you have a small set of calls that trigger a significant performance regression. This way you may be able to produce a small code example that triggers slow type checking.

[huguesb]

@Behoston you could try adding --timing-stats <path> to your mypy invocation. It will output a simple text file with per-module timings (one module per line). That should help narrow down which files are most impacted by the change, and from there you could further refine which statements in the specific files are most impacted.

[KelSolaar]

Hello,

We are experiencing similar issues with Colour: We went from ~30mins to ~1h 20mins on CI when updating our dependencies, one of them was mypy==0.961 --> mypy==0.982.

Cheers,

Thomas

[headtr1ck]

Our project also went from ~5min to ~25min going from 0.971 to 0.981/2.

[hauntsaninja]

It would be useful if you could post the details of your projects, so we can reproduce the slow down. As is mentioned above, it's very hard to diagnose these things in the abstract.

If your project is not open source, it would be very helpful if you could make a shareable repro of the bad behaviour. For example, try some of the strategies mentioned above:

• Use --timing-stats to narrow down which module is slow
• Potentially look for invocations of sum (since that's related to OP's problem, lots of thanks to Behoston for narrowing that down)