Fix weird docker error

[SkafteNicki]

What does this PR do?

Fixes #2559

I have no idea what the actual issue is here. It is therefore up for discussion if this PR should be merged.
Apparently, using torchmetrics inside a docker image can lead to the following error:

RuntimeError:
DataLoader worker (pid(s) 104) exited unexpectedly 

which can happen due to so many reasons in my experience. See the issue for more details, but I was able to reproduce the issue on v1.4.0 of torchmetrics but not on v1.3.2, thus after running git bisect I was able to narrow it down to this commit cd7ccfc. Then removing each line one by one I could narrow it down to the data loader having some weird interaction with the requirement cache.

I do not really think this is anything we can test for.

Before submitting

• Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

---

📚 Documentation preview 📚: https://torchmetrics--2581.org.readthedocs.build/en/2581/

[Borda]

Looks strange but if it helps...

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69%. Comparing base (f1a2be7) to head (6e1e877).
Report is 95 commits behind head on master.

Additional details and impacted files

@@          Coverage Diff           @@
##           master   #2581   +/-   ##
======================================
  Coverage      69%     69%           
======================================
  Files         314     314           
  Lines       17695   17696    +1     
======================================
+ Hits        12197   12198    +1     
  Misses       5498    5498