Skip to content

Build linux/macos arm64/aarch64 wheels on native runners #1308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ddelange wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Build linux/macos arm64/aarch64 wheels on native runners #1308

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7b5ce07
Build linux/macos arm64/aarch64 wheels on native runners
ddelange Feb 22, 2026
4015245
Replace pkg_resources with packaging
ddelange Feb 22, 2026
3bca715
Various CI speedups
ddelange Feb 22, 2026
613099e
Pin runner images to avoid CI rot
ddelange Feb 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 19 additions & 13 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:

jobs:
validate-release-request:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- name: Validate release PR
uses: edgedb/action-release/validate-pr@master
Expand Down Expand Up @@ -44,7 +44,7 @@ jobs:

build-sdist:
needs: validate-release-request
runs-on: ubuntu-latest
runs-on: ubuntu-24.04

env:
PIP_DISABLE_PIP_VERSION_CHECK: 1
Expand All @@ -63,8 +63,8 @@ jobs:

- name: Build source distribution
run: |
pip install -U setuptools wheel pip
python setup.py sdist
pip install -U setuptools wheel build
python -m build --sdist

- uses: actions/upload-artifact@v4
with:
Expand All @@ -73,7 +73,7 @@ jobs:

build-wheels-matrix:
needs: validate-release-request
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
outputs:
include: ${{ steps.set-matrix.outputs.include }}
steps:
Expand All @@ -83,14 +83,20 @@ jobs:
- uses: actions/setup-python@v6
with:
python-version: "3.x"
- run: pip install cibuildwheel==3.3.0
- run: pip install cibuildwheel==3.3.1
- id: set-matrix
env:
# skipping pypy for now
CIBW_SKIP: >
pp*
run: |
MATRIX_INCLUDE=$(
{
cibuildwheel --print-build-identifiers --platform linux --archs x86_64,aarch64 | grep cp | jq -nRc '{"only": inputs, "os": "ubuntu-latest"}' \
&& cibuildwheel --print-build-identifiers --platform macos --archs x86_64,arm64 | grep cp | jq -nRc '{"only": inputs, "os": "macos-latest"}' \
&& cibuildwheel --print-build-identifiers --platform windows --archs x86,AMD64 | grep cp | jq -nRc '{"only": inputs, "os": "windows-latest"}'
cibuildwheel --print-build-identifiers --platform linux --archs x86_64 | jq -nRc '{"only": inputs, "os": "ubuntu-24.04"}' \
&& cibuildwheel --print-build-identifiers --platform linux --archs aarch64 | jq -nRc '{"only": inputs, "os": "ubuntu-24.04-arm"}' \
&& cibuildwheel --print-build-identifiers --platform macos --archs x86_64 | jq -nRc '{"only": inputs, "os": "macos-15-intel"}' \
&& cibuildwheel --print-build-identifiers --platform macos --archs arm64 | jq -nRc '{"only": inputs, "os": "macos-15"}' \
&& cibuildwheel --print-build-identifiers --platform windows --archs x86,AMD64 | jq -nRc '{"only": inputs, "os": "windows-2025"}'
} | jq -sc
)
echo "include=$MATRIX_INCLUDE" >> $GITHUB_OUTPUT
Expand Down Expand Up @@ -123,7 +129,7 @@ jobs:
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0

- uses: pypa/cibuildwheel@63fd63b352a9a8bdcc24791c9dbee952ee9a8abc # v3.3.0
- uses: pypa/cibuildwheel@298ed2fb2c105540f5ed055e8a6ad78d82dd3a7e # v3.3.1
with:
only: ${{ matrix.only }}
env:
Expand All @@ -135,7 +141,7 @@ jobs:
path: wheelhouse/*.whl

merge-artifacts:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
needs: [build-sdist, build-wheels]
steps:
- name: Merge Artifacts
Expand All @@ -146,7 +152,7 @@ jobs:

publish-docs:
needs: [build-sdist, build-wheels]
runs-on: ubuntu-latest
runs-on: ubuntu-24.04

env:
PIP_DISABLE_PIP_VERSION_CHECK: 1
Expand Down Expand Up @@ -195,7 +201,7 @@ jobs:

publish:
needs: [build-sdist, build-wheels, publish-docs]
runs-on: ubuntu-latest
runs-on: ubuntu-24.04

environment:
name: pypi
Expand Down
61 changes: 30 additions & 31 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ on:
branches:
- master

concurrency: # https://stackoverflow.com/questions/66335225#comment133398800_72408109
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
test-platforms:
# NOTE: this matrix is for testing various combinations of Python and OS
Expand All @@ -18,12 +22,15 @@ jobs:
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t"]
os: [ubuntu-latest, macos-latest, windows-latest]
os: [ubuntu-24.04, macos-15, windows-2025]
loop: [asyncio, uvloop]
exclude:
# uvloop does not support windows
- loop: uvloop
os: windows-latest
os: windows-2025
# github does not provide 3.9 cpython for arm64 runners
- python-version: "3.9"
os: macos-15
Comment on lines +31 to +33
Copy link
Contributor Author

@ddelange ddelange Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi Python 3.9 is EOL since 31 Oct 2025, can consider dropping support


runs-on: ${{ matrix.os }}

Expand All @@ -33,9 +40,6 @@ jobs:
run:
shell: bash

env:
PIP_DISABLE_PIP_VERSION_CHECK: 1

steps:
- uses: actions/checkout@v5
with:
Expand All @@ -53,50 +57,44 @@ jobs:
version_line_pattern: |
__version__(?:\s*:\s*typing\.Final)?\s*=\s*(?:['"])([[:PEP440:]])(?:['"])

- name: Setup PostgreSQL
if: "!steps.release.outputs.is_release && matrix.os == 'macos-latest'"
- name: Set up PostgreSQL
if: "!steps.release.outputs.is_release && startsWith(matrix.os, 'macos')"
run: |
POSTGRES_FORMULA="postgresql@18"
brew install "$POSTGRES_FORMULA"
echo "$(brew --prefix "$POSTGRES_FORMULA")/bin" >> $GITHUB_PATH

- name: Set up Kerberos
if: "!steps.release.outputs.is_release && startsWith(matrix.os, 'ubuntu')"
run: .github/workflows/install-krb5.sh

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
uses: astral-sh/setup-uv@v7
if: "!steps.release.outputs.is_release"
with:
python-version: ${{ matrix.python-version }}
activate-environment: true

- name: Install Python Deps
if: "!steps.release.outputs.is_release"
run: |
[ "$RUNNER_OS" = "Linux" ] && .github/workflows/install-krb5.sh
python -m pip install -U pip setuptools wheel
python -m pip install --group test
python -m pip install -e .
uv pip install -e . --group test

- name: Test
if: "!steps.release.outputs.is_release"
env:
LOOP_IMPL: ${{ matrix.loop }}
run: |
if [ "${LOOP_IMPL}" = "uvloop" ]; then
env USE_UVLOOP=1 python -m unittest -v tests.suite
else
python -m unittest -v tests.suite
fi
USE_UVLOOP: ${{ matrix.loop == 'uvloop' && '1' || '' }}
run: python -m unittest -v tests.suite

test-postgres:
strategy:
matrix:
postgres-version: ["9.5", "9.6", "10", "11", "12", "13", "14", "15", "16", "17", "18"]

runs-on: ubuntu-latest
runs-on: ubuntu-24.04

permissions: {}

env:
PIP_DISABLE_PIP_VERSION_CHECK: 1

steps:
- uses: actions/checkout@v5
with:
Expand Down Expand Up @@ -125,19 +123,20 @@ jobs:
echo PGINSTALLATION="/usr/lib/postgresql/${PGVERSION}/bin" \
>> "${GITHUB_ENV}"

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
- name: Set up Kerberos
if: "!steps.release.outputs.is_release"
run: .github/workflows/install-krb5.sh

- name: Set up Python
uses: astral-sh/setup-uv@v7
if: "!steps.release.outputs.is_release"
with:
python-version: "3.x"
activate-environment: true

- name: Install Python Deps
if: "!steps.release.outputs.is_release"
run: |
[ "$RUNNER_OS" = "Linux" ] && .github/workflows/install-krb5.sh
python -m pip install -U pip setuptools wheel
python -m pip install --group test
python -m pip install -e .
uv pip install -e . --group test

- name: Test
if: "!steps.release.outputs.is_release"
Expand All @@ -151,7 +150,7 @@ jobs:
regression-tests:
name: "Regression Tests"
needs: [test-platforms, test-postgres]
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
permissions: {}

steps:
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ docs = [
[build-system]
requires = [
"setuptools>=77.0.3",
"Cython(>=3.2.1,<4.0.0)"
"Cython>=3.2.1,<4.0.0"
]
build-backend = "setuptools.build_meta"

Expand Down
9 changes: 5 additions & 4 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
from setuptools.command import build_ext as setuptools_build_ext


CYTHON_DEPENDENCY = 'Cython(>=3.2.1,<4.0.0)'
CYTHON_DEPENDENCY = 'Cython>=3.2.1,<4.0.0'

CFLAGS = ['-O2']
LDFLAGS = []
Expand Down Expand Up @@ -186,7 +186,8 @@ def finalize_options(self):
need_cythonize = True

if need_cythonize:
import pkg_resources
# 'packaging' gets installed by 'wheel'
from packaging.requirements import Requirement

# Double check Cython presence in case setup_requires
# didn't go into effect (most likely because someone
Expand All @@ -199,8 +200,8 @@ def finalize_options(self):
'please install {} to compile asyncpg from source'.format(
CYTHON_DEPENDENCY))

cython_dep = pkg_resources.Requirement.parse(CYTHON_DEPENDENCY)
if Cython.__version__ not in cython_dep:
cython_dep = Requirement(CYTHON_DEPENDENCY)
if Cython.__version__ not in cython_dep.specifier:
raise RuntimeError(
'asyncpg requires {}, got Cython=={}'.format(
CYTHON_DEPENDENCY, Cython.__version__
Expand Down
Loading