BLD/RLS: Update wheels to include GDAL 3.9.2 and libkml #451
Conversation
|
Adding LIBKML appears to require a number of boost libraries, which may make the wheels a bit bigger: * boost-assert:arm64-linux@1.85.0#1
* boost-cmake:arm64-linux@1.85.0#1
* boost-config:arm64-linux@1.85.0#1
* boost-core:arm64-linux@1.85.0#2
* boost-headers:arm64-linux@1.85.0#1
* boost-move:arm64-linux@1.85.0#1
* boost-smart-ptr:arm64-linux@1.85.0#1
* boost-static-assert:arm64-linux@1.85.0#1
* boost-throw-exception:arm64-linux@1.85.0#1
* boost-type-traits:arm64-linux@1.85.0#1
* boost-uninstall:arm64-linux@1.85.0#1
libkml:arm64-linux@1.3.0#13
* minizip:arm64-linux@1.3.1#1
* uriparser:arm64-linux@0.9.8
* vcpkg-boost:arm64-linux@2024-05-15spatialite is creating some dependency issues: it brings in freexl by default, which is failing to build. I don't know that we actually need it to support spatialite functionality in GDAL / pyogrio. (need to disable default features for spatialite) |
|
I can't seem to get libspatialite to build within vcpkg locally, even if I just try to build it directly and disable default features. It appears to be failing to detect sqlite3, even though that gets installed properly by vcpkg. Unless anyone has some brilliant ideas how to get this working, I think we should leave libspatialite out for now. |
|
We appear to have jobs stuck in the github actions queue for a long time (due to github actions outage / issues yesterday), and canceling them does not clear them out. I'm going to temporarily close this PR to see if that triggers github to clear out those jobs. |
…nning tests, temporarily disable some jobs
brendan-ward
changed the title
|
Since it looks like this PR is needed for Python 3.13 wheels, what's needed to move it forward? |
|
GDAL 3.9.2 was released with additional bug fixes. |
brendan-ward
changed the title
|
Failures using Conda appear related to shapely #2098, which will be resolved when shapely 2.0.6 lands on conda-forge. |
|
A nice badge to view when Shapely 2.0.6 is available on conda-forge: |
|
The following PR will avoid the conda environment to be cached (for a very long time, based on some experiences in other projects typically till the relevant env.yml file is changed). This will lead to shapely 2.0.6 to be actually used in the tests: #461. |
|
Looks fully green! |
.github/workflows/release.yml
Outdated
| . $HOME/.cargo/env | ||
| uv venv .venv | ||
| echo "VIRTUAL_ENV=.venv" >> $GITHUB_ENV | ||
| echo "$PWD/.venv/bin" >> $GITHUB_PATH |
There was a problem hiding this comment.
What is this doing exactly? (is it common practice documented somewhere, or can we add some comments?)
Is it to ensure the python on the path is the one of this venv?
From quickly checking the uv docs (https://docs.astral.sh/uv/pip/environments/), my understanding is that when you use the default .venv name for the environment, then subsequent uv pip .. automatically uses that environment
There was a problem hiding this comment.
But I suppose it is for the python3 -m pytest .. below? That could maybe also use uv run python -m pytest ... to make it explicit it is using the python from the uv venv?
There was a problem hiding this comment.
Correct, I believe I was running into issues with uv not automatically activating the virtual environment in subsequent steps. Somehow I missed that you could issue uv run python ... instead; will try that now.
There was a problem hiding this comment.
Just dropping a note that trying uv run python did not work in this case (appears to require [project] section be added to pyproject.toml plus a few more things I didn't investigate fixing); it was easier to just use the environment variables to activate the virtual environment.
|
|
||
| steps: | ||
| - name: Install packages | ||
| run: | | ||
| apt-get update && apt-get install -y build-essential python3-dev |
There was a problem hiding this comment.
Do you know why this is now needed? (was there a change in the docker image?)
There was a problem hiding this comment.
Originally we were installing python3-pip which also pulled in build-essential and python3-dev as dependencies. We now need to install them directly.
| @@ -178,7 +186,7 @@ jobs: | |||
| git reset --hard | |||
| # pull latest version to ensure the required commit with GDAL 3.8 is available | |||
| git pull | |||
| git checkout 4f4a1821b2e8c7a2863e4df65a4d514f84144049 | |||
| git checkout 73794ce5f63fd138fab999a22959ca7c6305d93c | |||
|
|
|||
| - name: Install GDAL | |||
| env: | |||
There was a problem hiding this comment.
There is a --overlay-triplets=./ci/custom-triplets a few lines below here that can then be removed as well
There was a problem hiding this comment.
Don't we still need to use the custom triplets?
|
Superseded by #466 |
(edit: changed 3.9.1 to 3.9.2)
resolves #447, #453, #454
Uses GDAL 3.9.2 from VCPKG, which requires bumping this to the next minor release (0.10.0), and enables the libkml features (resolves #447). Also adds tests for GDAL 3.9.2 to CI.
This drops use of manylinux2014 and uses manylinux_2_28 (minimum recommended version) for x86_64 builds (resolves #453); we were already using it for Arm64 builds. This drops the custom port of zlib, which no longer appears necessary (was due to version conflicts with manylinux2014) and was causing issues adding one of the dependencies of libkml. Switching to manylinux_2_28 also obviates the build error we've been seeing lately due to CentOS 7 (used by manylinux2014) EOL.
This switches to pulling the Arm64 MacOS wheel when testing on the
macos-latestrunner, which is MacOS / Arm64 (resolves #454) and uses--no-indexflag touv pip installto ensure it doesn't silently fall back to using an Arm64 wheel from PyPI.This switches to
uvfor installing Python dependencies, in line with #438, which also gets around issues with Python 3.12 requiring a virtual environment (needed for testing the sdist using thegdal:ubuntu-small-3.9.2base image which has Python 3.12).I initially tried to add libspatialite, but was unsuccessful and getting that to build within VCPKG.