Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TEST-#7166: Fix HDF tests in CI #7167

Merged
merged 2 commits into from
Apr 11, 2024
Merged

TEST-#7166: Fix HDF tests in CI #7167

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
88addf9
TEST-#7166: Fix HDF tests in CI
anmyachev Apr 10, 2024
105ed3c
pin c-blosc2<=2.14.1
anmyachev Apr 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
pin c-blosc2<=2.14.1 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
  • Loading branch information
@anmyachev
anmyachev committed Apr 10, 2024
commit 105ed3c62751fb53c414e59ba3d428e25a9a72d4
12 changes: 6 additions & 6 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ jobs:
with:
environment-file: environment-dev.yml
- name: Install HDF5
run: sudo apt update && sudo apt install -y libhdf5-dev libfuse2
run: sudo apt update && sudo apt install -y libhdf5-dev
- name: xgboost tests
run: |
# TODO(https://github.com/modin-project/modin/issues/5194): Uncap xgboost
Expand Down Expand Up @@ -175,7 +175,7 @@ jobs:
environment-file: requirements/env_hdk.yml
activate-environment: modin_on_hdk
- name: Install HDF5
run: sudo apt update && sudo apt install -y libhdf5-dev libfuse2
run: sudo apt update && sudo apt install -y libhdf5-dev
- run: python -m pytest modin/tests/core/storage_formats/hdk/test_internals.py
- run: python -m pytest modin/tests/experimental/hdk_on_native/test_init.py
- run: python -m pytest modin/tests/experimental/hdk_on_native/test_dataframe.py
Expand Down Expand Up @@ -347,7 +347,7 @@ jobs:
activate-environment: modin_on_unidist
python-version: ${{matrix.python-version}}
- name: Install HDF5
run: sudo apt update && sudo apt install -y libhdf5-dev libfuse2
run: sudo apt update && sudo apt install -y libhdf5-dev
- name: Set up postgres
# Locally, specifying port 2345:5432 works, but 2345:2345 and 5432:5432 do not. This solution is from
# https://stackoverflow.com/questions/36415654/cant-connect-docker-postgresql-9-3
Expand Down Expand Up @@ -451,7 +451,7 @@ jobs:
command: ray start --head --port=6379 --object-store-memory=1000000000
if: matrix.os == 'windows' && matrix.engine == 'ray'
- name: Install HDF5
run: sudo apt update && sudo apt install -y libhdf5-dev libfuse2
run: sudo apt update && sudo apt install -y libhdf5-dev
if: matrix.os == 'ubuntu'
- name: Set up postgres
# Locally, specifying port 2345:5432 works, but 2345:2345 and 5432:5432 do not. This solution is from
Expand Down Expand Up @@ -564,7 +564,7 @@ jobs:
activate-environment: ${{ matrix.execution.name == 'unidist' && 'modin_on_unidist' || 'modin' }}
python-version: ${{matrix.python-version}}
- name: Install HDF5
run: sudo apt update && sudo apt install -y libhdf5-dev libfuse2
run: sudo apt update && sudo apt install -y libhdf5-dev
if: matrix.os != 'windows'
- name: Limit ray memory
run: echo "MODIN_MEMORY=1000000000" >> $GITHUB_ENV
Expand Down Expand Up @@ -677,7 +677,7 @@ jobs:
with:
environment-file: environment-dev.yml
- name: Install HDF5
run: sudo apt update && sudo apt install -y libhdf5-dev libfuse2
run: sudo apt update && sudo apt install -y libhdf5-dev
- run: python -m pytest -n 2 modin/tests/pandas/dataframe/test_map_metadata.py
- run: python -m pytest -n 2 modin/tests/pandas/test_series.py
# Do not add parallelism (`-n` argument) here - it will cause mock S3 service to fail.
Expand Down
1 change: 1 addition & 0 deletions environment-dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ dependencies:
- sqlalchemy>=2.0.0
- pandas-gbq>=0.19.0
- pytables>=3.8.0
- c-blosc2<=2.14.1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also update requirements-dev.txt?

Copy link
Collaborator Author

@anmyachev anmyachev Apr 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is not necessary, we just need the tests in CI to work as quickly as possible, I believe that either pytables or c-blosc2 will release their new patches to make everything work (fast enough). Also, I'm not sure if this error is present when installing using pip.

# pymssql==2.2.8 broken: https://github.com/modin-project/modin/issues/6429
- pymssql>=2.1.5,!=2.2.8
- psycopg2>=2.9.6
Expand Down
1 change: 1 addition & 0 deletions requirements/env_hdk.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ dependencies:
- matplotlib>=3.6.3
- xarray>=2022.12.0
- pytables>=3.8.0
- c-blosc2<=2.14.1
- fastparquet>=2022.12.0
# pandas isn't compatible with numexpr=2.8.5: https://github.com/modin-project/modin/issues/6469
- numexpr<2.8.5
Expand Down
1 change: 1 addition & 0 deletions requirements/env_unidist_linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ dependencies:
- sqlalchemy>=2.0.0
- pandas-gbq>=0.19.0
- pytables>=3.8.0
- c-blosc2<=2.14.1
# pymssql==2.2.8 broken: https://github.com/modin-project/modin/issues/6429
- pymssql>=2.1.5,!=2.2.8
- psycopg2>=2.9.6
Expand Down
1 change: 1 addition & 0 deletions requirements/env_unidist_win.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ dependencies:
- sqlalchemy>=2.0.0
- pandas-gbq>=0.19.0
- pytables>=3.8.0
- c-blosc2<=2.14.1
# pymssql==2.2.8 broken: https://github.com/modin-project/modin/issues/6429
- pymssql>=2.1.5,!=2.2.8
- psycopg2>=2.9.6
Expand Down
1 change: 1 addition & 0 deletions requirements/requirements-no-engine.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ dependencies:
- sqlalchemy>=2.0.0
- pandas-gbq>=0.19.0
- pytables>=3.8.0
- c-blosc2<=2.14.1
- tqdm>=4.60.0
# pandas isn't compatible with numexpr=2.8.5: https://github.com/modin-project/modin/issues/6469
- numexpr<2.8.5
Expand Down
Loading