CI: update testing matrix for Python versions (3.12 & 3.13) (#589)

* update CI testing matrix to include Python 3.12 and 3.13

* increasing the timeout time , but not sure why the new versions are taking more time

* Update .github/workflows/ci-testing.yml

* Apply suggestions from code review

* test few updates to ci

* update

* revert changes

* with pytest-cov

* bump

* ex Win

* cov

* --capture=tee-sys --exitfirst

* PYTHONMULTIPROCESSING_SPAWN: "spawn"

* --skip=processing --skip=streaming

* --ignore=processing --ignore=streaming

* --ignore=streaming

* --ignore=processing

* Update .github/workflows/ci-testing.yml

* --ignore=processing/functions.py

* ignore processor and functions from processing

* ignore readers

* ignore correct files

* let's just keep data processor now

* ignore macos for higher versions

* check for : ignore test_functions.py

* skip a few tests under functions.py

* ignore other than `test_functions` to speedup

* fix: correct typo in ignore flag for pytest command

* ignore few more

* ignore more

* temporarily skip multiple tests in test_functions.py

* remove temporary skip from test_map_with_path

* remove temporary skip from test_optimize_with_jpeg_array

* just keep skip for: test_optimize_checkpoint_in_none_and_append_mode

* update test_optimize_checkpoint_in_none_and_append_mode to skip based on Python version and platform

* remove test ignores from CI workflow

---------

Co-authored-by: Deependu <deependujha21@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka B <j.borovec+github@gmail.com>

Author: 3 people

.github/workflows/ci-testing.yml

@@ -19,10 +19,14 @@ jobs:
       fail-fast: false
       matrix:
         os: ["ubuntu-22.04", "macos-14", "windows-2022"]
-        python-version: ["3.9", "3.10", "3.11"]
+        python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
+        exclude:
+          - { os: "windows-2022", python-version: "3.13" }
+          - { os: "macos-14", python-version: "3.12" }
+          - { os: "macos-14", python-version: "3.13" }
 
     # Timeout: https://stackoverflow.com/a/59076067/4521646
-    timeout-minutes: 45
+    timeout-minutes: 60
     env:
       TORCH_URL: "https://download.pytorch.org/whl/cpu/torch_stable.html"
 
@@ -42,9 +46,11 @@ jobs:
           pip list
 
       - name: Tests
-        run: coverage run --source litdata -m pytest tests -v --durations=100
+        working-directory: tests
+        run: pytest . -v --cov=litdata --durations=100
 
       - name: Statistics
+        continue-on-error: true
         run: |
           coverage report
           coverage xml

requirements/test.txt

@@ -1,8 +1,8 @@
-coverage ==7.6.*
+coverage ==7.8.*
 cryptography==42.0.8
 mosaicml-streaming==0.11.0
 pytest ==8.4.*
-pytest-cov ==5.0.0
+pytest-cov ==6.1.1
 pytest-timeout ==2.3.1
 pytest-rerunfailures ==14.0
 pytest-random-order ==1.1.1

tests/processing/test_functions.py

@@ -225,6 +225,7 @@ def test_optimize_append_overwrite(tmpdir):
     assert ds[:] == [(i, i**2, i**3) for i in range(0, 5)]
 
 
+@pytest.mark.skipif(sys.version_info >= (3, 12), reason="Multiprocessing issues on Python 3.12+")
 @pytest.mark.skipif(sys.platform == "win32", reason="too slow")
 def test_optimize_checkpoint_in_none_and_append_mode(tmpdir):
     output_dir = str(tmpdir / "output_dir")