Repository navigation
repeating a test spams pytest cache #84
Description
When running many repeatitions of a test with --count, .pytest_cache/v/cache/nodeids may get up to millions of entries. Eg:
"test/utils/test_types.py::test_if_none[1-100]",
"test/utils/test_types.py::test_if_none[10-100]",
"test/utils/test_types.py::test_if_none[100-100]",
"test/utils/test_types.py::test_if_none[11-100]",
"test/utils/test_types.py::test_if_none[12-100]",
...
This may cause a lag of several seconds even after executing a single method with -k specifying a single test file.
The culprit is pytest_sessionfinish in cacheprovider.py, when calling config.cache.set("cache/nodeids", sorted(self.cached_nodeids)). When cached_nodeids contains milions of entries, this function always generates a json with the whole contents of .pytest_cache/v/cache/nodeids.
Activity
RonnyPfannschmidt commentedon Apr 14, 2024
Please elaborate on the actual numbers
dpinol commentedon Apr 14, 2024
The length of ".pytest_cache/v/cache/nodeids" depends on the usage. I guess I executed at some time with millions of executions and got the file with so many lines. In that case, on my intel i9 a trivial test took 4s, while after removing the file the whole execution now takes 0.3s
dpinol commentedon Apr 14, 2024
I tried to reproduce it. I executed 1M times a file with 2 tests. It created a cache file of 143M with 2M entries.
Executing the tests a single time passed from 0.4s to 2.4s
RonnyPfannschmidt commentedon Apr 14, 2024
as repeat uses parameterize - i suspect a breaking change is needed to make this more usable
okken commentedon Apr 30, 2024
You can disable caching entirely while running large counts with
-p no:cacheprovider.Would that work?
okken commentedon Feb 16, 2025
Recommend closing this issue, as there's not much this plugin can do about it.
dpinol commentedon Feb 17, 2025
Sorry for the late answer.
-p no:cacheproviderworks like a charm, thanks!I guess users would appreciate a note in the documentation about this issue, probably mentioning the
-p no:cacheprovidertrick. I added it to my pytest wrapper script when it detects--countsince I guess the cache make not much sense when repeating a tests.dmakhno commentedon Mar 26, 2025
Is it related to when
pytest_generate_testsis executed?Lmk if my case is another one, and I'll create separate issue.
I have a suite, let's say with 10000 tests.
And I have -k or -m which selects among them only 10.
And e.g. I'm using --count 10
What I'd expected:
"Run 100, Deselected 9990"
What I have:
"Run 100, Deselected 99900" - tests were multiplied before filtering
Is there way to postpone generate hook or "multiplication" call later somehow?