Slighy faster speed

[tchaton]

Before submitting

• Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

Before:

prefetch_factor = 10
⚡ ~ python benchmark/stream_imagenet.py
Seed set to 42
Benchmarking using litdata version: 0.2.41
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5005/5005 [04:13<00:00, 19.76it/s]
For /teamspace/studios/this_studio/benchmark/stream_imagenet.py on 0, streamed over 1281167 samples in 253.43733644485474 or 5055.162747104145 images/sec.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5005/5005 [03:55<00:00, 21.26it/s]
For /teamspace/studios/this_studio/benchmark/stream_imagenet.py on 1, streamed over 1281167 samples in 235.43075132369995 or 5441.799668172387 images/sec.

prefetch_factor = 2

⚡ ~ python benchmark/stream_imagenet.py
Seed set to 42
Benchmarking using litdata version: 0.2.41
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5005/5005 [03:59<00:00, 20.86it/s]
For /teamspace/studios/this_studio/benchmark/stream_imagenet.py on 0, streamed over 1281167 samples in 240.06360864639282 or 5336.781303894049 images/sec.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5005/5005 [03:49<00:00, 21.83it/s]
For /teamspace/studios/this_studio/benchmark/stream_imagenet.py on 1, streamed over 1281167 samples in 229.28257656097412 or 5587.720627958314 images/sec.
Finished benchmarking.

What does this PR do?

Fixes # (issue).

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

Attention: Patch coverage is 62.22222% with 17 lines in your changes missing coverage. Please review.

Project coverage is 79%. Comparing base (a2533b2) to head (3d6080e).
Report is 1 commits behind head on main.

Additional details and impacted files

@@         Coverage Diff         @@
##           main   #503   +/-   ##
===================================
- Coverage    79%    79%   -0%     
===================================
  Files        38     38           
  Lines      5691   5719   +28     
===================================
+ Hits       4509   4525   +16     
- Misses     1182   1194   +12     

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.