Repository navigation
Add Windows ARM64 build - #1937
Conversation
|
Thanks for this, @ndabas — and welcome! This is a well-scoped first contribution, and you've correctly identified the two subtle traps that make Windows-on-ARM builds go wrong, which most people miss:
A few things I'd like to confirm or tighten already at this point — mostly about making the native-ARM64 guarantee enforced rather than best-effort, since the failure mode here is silent (mis-tagged wheels), which is the worst kind IMO: 1. Make the native-arch check a hard failure. The emulation guard is the crux of the whole PR. Please have the build assert the interpreter's 2. Confirm the wheels are actually tested on the native runner, not just built. Building isn't proof. Please run at least an import smoke test on 3. Scope: CPython-only. PyPy has no Windows/ARM64 build, so the 4. Track the temporary OpenSSL/cryptography step for removal. Good that you flagged it as temporary — let's not lose it: gate it on the 5. Update the stale note + the platform docs. 6. Wheel tag + repair. Please confirm the emitted wheel is tagged 7. Where it sits in our wheel tiers. We're formalizing supported-vs-best-effort wheel tiers (#1935). Given a brand-new platform + the temporary crypto bootstrap + runner availability, I'd slot CI is still running as I write this — I'll do a final pass once it's green, and in particular I want to see in the logs that the (And thanks for including the AI-assistance disclosure per our |
|
@ndabas — quick heads-up on the one red check: "Code Quality Checks" is on us, not you. That job runs I've opened #1938 to remove that dead code on Thanks again for the contribution, and apologies for the red ✗ that isn't yours. |
|
Thanks for the review, and the clarification on the failing CI job, @oberstet! I don't think I've changed anything that should touch or cause the "wheels-arm64 / Build ARM64 wheels (pypy-3.11-bookworm-manylinux_2_36_aarch64" job to fail, but I'll double-check. On the points you have noted, 3 and 5 are already done. For the rest I'll verify and add whatever is missing and let you know. |
|
Thanks again for the great review. On the remaining points:
On the other failing job ("Build ARM64 wheels (pypy-3.11-bookworm-manylinux_2_36_aarch64)"), it looks like the current release of |
* start new dev branch for #1938 * Remove dead code surfaced by ty (#1938) Delete three pieces of pre-existing dead code that ty (floated on the bleeding edge) flagged on the CI run for #1937, rather than growing the ty-ignore list: - src/autobahn/asyncio/component.py: unused private helper _unique_list (no call sites anywhere in the repo). - src/autobahn/twisted/component.py: identical unused _unique_list. - src/autobahn/websocket/protocol.py: the `if response_body:` block in the server opening handshake was unreachable - response_body is hard-set to None and never reassigned, and a 101 Switching Protocols response carries no body. Removed the block and the dead assignment. Pure deletion (24 lines); all private/unreachable, no public API change. Verified locally: check-typing clean (ruff + ty, all checks passed); Twisted 357 tests PASSED (325 successes, 32 skips); asyncio 275 passed, 0 failures. Handshake output is unchanged (the removed block never ran). Full wstest compliance + Crossbar smoke run in CI. Note: This work was completed with AI assistance (Claude Code).
|
Thanks @ndabas — these are great responses; you addressed every point, and a couple are better than what I asked for. Where I've landed: Please rebase on
On the individual points:
After the rebase it should run green — with one known exception: the emulated Really nice contribution — thanks for the care on the WoA emulation traps. |
|
Thank you for the clear responses -- I have updated the code to match. I used the existing recipes for the smoke tests, and added a couple of tests there. Let me know if that looks okay. CI is all green on my fork now, thanks for pushing those fixes so quickly. On the wheel tiers, the On a side note, we could consolidate much of the Windows-specific/other platform-specific code since most of those are doing exactly the same thing on every platform. Both |
|
thanks a lot! "windows on arm64" coming. side question: are you only after autobahn-python, or zlmdb or crossbar as well for win/arm64? also, I took the chance to open issues to improve docs as well now: #1946 (and crossbario/zlmdb#147, and crossbario/crossbar#2287) |
|
@ndabas : https://github.com/crossbario/autobahn-python/releases/tag/master-202609230557 this now includes win_arm64 wheels! for cpython. if you have a chance to actually try these on your side, would be curious to know! I don't have any win machine ... 
|
|
Excellent, thanks @oberstet! I tested the nightly on my Windows ARM64 laptop, works fine -- here's the script I used to do that here, in case someone wants to use it (because we need that OpenSSL build for now until <#
.SYNOPSIS
Install and verify autobahn on native Windows ARM64 (win_arm64).
.DESCRIPTION
autobahn publishes win_arm64 wheels, but its `cryptography` dependency does not:
no cryptography release since 46.0.3 ships a win_arm64 wheel, and autobahn
requires cryptography>=50. pip therefore has to build cryptography from source,
which needs Rust, the MSVC ARM64 toolchain and OpenSSL headers/libs.
This script provisions OpenSSL via vcpkg, creates a clean virtual environment,
installs the autobahn wheel from a GitHub release, and runs a set of functional
checks that do not depend on the autobahn source tree.
Everything it creates is confined to -VcpkgRoot and -VenvPath. It does not modify
the machine-wide environment; OPENSSL_DIR is set for this process only.
.PARAMETER ReleaseTag
Tag of the GitHub release to pull the wheel from.
.PARAMETER PythonVersion
CPython feature version to install into the venv (3.11 - 3.14).
.PARAMETER VenvPath
Directory for the throwaway virtual environment.
.PARAMETER VcpkgRoot
Directory for the vcpkg checkout. Reused if it already exists.
.PARAMETER PythonExe
Optional path to an existing native ARM64 python.exe. If omitted, uv is used to
fetch a matching interpreter (recommended).
.EXAMPLE
.\install-win-arm64.ps1
.EXAMPLE
.\install-win-arm64.ps1 -PythonVersion 3.12 -VenvPath C:\venvs\autobahn
#>
[CmdletBinding()]
param(
[string]$ReleaseTag = 'master-202609230557',
[ValidateSet('3.11', '3.12', '3.13', '3.14')]
[string]$PythonVersion = '3.14',
[string]$VenvPath = (Join-Path $env:TEMP 'autobahn-win-arm64'),
[string]$VcpkgRoot = (Join-Path $env:USERPROFILE 'vcpkg'),
[string]$PythonExe
)
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'SilentlyContinue' # Invoke-WebRequest is ~10x slower with the progress bar
$Repo = 'crossbario/autobahn-python'
$Triplet = 'arm64-windows-static-md'
function Write-Step { param([string]$Message) Write-Host "`n==> $Message" -ForegroundColor Cyan }
function Write-Info { param([string]$Message) Write-Host " $Message" -ForegroundColor DarkGray }
function Die { param([string]$Message) Write-Host "`nERROR: $Message" -ForegroundColor Red; exit 1 }
# ---------------------------------------------------------------- preflight --
Write-Step 'Checking prerequisites'
if ($env:PROCESSOR_ARCHITECTURE -ne 'ARM64') {
Die "this script must run in a native ARM64 shell (found $env:PROCESSOR_ARCHITECTURE). If you are in an x64 PowerShell on an ARM64 device, launch the ARM64 build instead."
}
Write-Info "host architecture: ARM64"
if (-not (Get-Command git -ErrorAction SilentlyContinue)) {
Die 'git not found. Install Git for Windows: winget install Git.Git'
}
# cryptography is a Rust extension; it needs cargo and the ARM64 target.
if (-not (Get-Command cargo -ErrorAction SilentlyContinue)) {
Die 'Rust not found; cryptography must be compiled from source. Install it with: winget install Rustlang.Rustup'
}
Write-Info "rust: $((rustc --version) -join '')"
# openssl-sys and cryptography's cdylib both need the MSVC ARM64 compiler.
$vswhere = Join-Path ${env:ProgramFiles(x86)} 'Microsoft Visual Studio\Installer\vswhere.exe'
if (-not (Test-Path $vswhere)) {
Die 'Visual Studio Installer not found. Install VS Build Tools with the "MSVC v143 - VS 2022 C++ ARM64 build tools" component.'
}
$vsPath = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.ARM64 -property installationPath
if (-not $vsPath) {
Die 'MSVC ARM64 build tools not installed. Add the "MSVC v143 - VS 2022 C++ ARM64 build tools" component via the Visual Studio Installer.'
}
Write-Info "msvc: $vsPath"
if (-not $PythonExe -and -not (Get-Command uv -ErrorAction SilentlyContinue)) {
Die 'uv not found and -PythonExe not given. Install uv (winget install astral-sh.uv) or pass -PythonExe pointing at a native ARM64 python.exe'
}
# ------------------------------------------------------------------- vcpkg --
Write-Step "Provisioning OpenSSL via vcpkg (triplet: $Triplet)"
$vcpkgExe = Join-Path $VcpkgRoot 'vcpkg.exe'
if (-not (Test-Path $vcpkgExe)) {
if (-not (Test-Path $VcpkgRoot)) {
Write-Info "cloning vcpkg into $VcpkgRoot"
git clone --depth 1 https://github.com/microsoft/vcpkg.git $VcpkgRoot
if ($LASTEXITCODE -ne 0) { Die 'git clone of vcpkg failed' }
}
Write-Info 'bootstrapping vcpkg (downloads its own cmake/ninja)'
& (Join-Path $VcpkgRoot 'bootstrap-vcpkg.bat') -disableMetrics
if ($LASTEXITCODE -ne 0) { Die 'vcpkg bootstrap failed' }
}
Write-Info "vcpkg: $vcpkgExe"
$opensslDir = Join-Path $VcpkgRoot "installed\$Triplet"
if (Test-Path (Join-Path $opensslDir 'include\openssl\ssl.h')) {
Write-Info 'openssl already present, skipping build'
}
else {
Write-Info 'building openssl - this takes a while on first run'
& $vcpkgExe install --triplet $Triplet --clean-after-build openssl
if ($LASTEXITCODE -ne 0) { Die 'vcpkg install openssl failed' }
}
# Consumed by the openssl-sys crate during the cryptography build. Process-scoped.
$env:OPENSSL_DIR = $opensslDir
Write-Info "OPENSSL_DIR = $env:OPENSSL_DIR"
# -------------------------------------------------------------------- venv --
Write-Step "Creating a clean virtual environment ($VenvPath)"
if (Test-Path $VenvPath) {
Write-Info 'removing previous environment'
Remove-Item -Recurse -Force $VenvPath
}
if ($PythonExe) {
& $PythonExe -m venv $VenvPath
if ($LASTEXITCODE -ne 0) { Die "venv creation failed using $PythonExe" }
}
else {
# The platform suffix is required: on Windows ARM64 a bare "cpython-3.14"
# request resolves to an x86_64 interpreter running under emulation.
uv venv --seed --python "cpython-$PythonVersion-windows-aarch64-none" $VenvPath
if ($LASTEXITCODE -ne 0) { Die 'uv venv creation failed' }
}
$py = Join-Path $VenvPath 'Scripts\python.exe'
if (-not (Test-Path $py)) { Die "no interpreter at $py" }
$platform = & $py -c "import sysconfig; print(sysconfig.get_platform())"
if ($platform -ne 'win-arm64') {
Die "interpreter is '$platform', expected 'win-arm64'. An emulated x86_64 Python cannot load win_arm64 wheels."
}
$pyTag = & $py -c "import sys; print(f'cp{sys.version_info.major}{sys.version_info.minor}')"
Write-Info "interpreter: $platform ($pyTag)"
# ------------------------------------------------------------------- wheel --
Write-Step "Downloading the $pyTag win_arm64 wheel from release '$ReleaseTag'"
$headers = @{ 'User-Agent' = 'autobahn-win-arm64-installer' }
try {
$release = Invoke-RestMethod "https://api.github.com/repos/$Repo/releases/tags/$ReleaseTag" -Headers $headers
}
catch {
Die "could not read release '$ReleaseTag': $($_.Exception.Message)"
}
$asset = $release.assets | Where-Object { $_.name -like "*-$pyTag-$pyTag-win_arm64.whl" } | Select-Object -First 1
if (-not $asset) {
$have = ($release.assets | Where-Object { $_.name -like '*win_arm64*' } | ForEach-Object { $_.name }) -join ', '
Die "no $pyTag win_arm64 wheel in '$ReleaseTag'. Available: $(if ($have) { $have } else { '(none)' })"
}
$wheel = Join-Path $VenvPath $asset.name
Invoke-WebRequest -Uri $asset.browser_download_url -OutFile $wheel -Headers $headers
Write-Info ("{0} ({1:N0} KB)" -f $asset.name, ((Get-Item $wheel).Length / 1KB))
# ----------------------------------------------------------------- install --
Write-Step 'Installing the wheel (cryptography is compiled from source here)'
& $py -m pip install --upgrade pip setuptools wheel --quiet
if ($LASTEXITCODE -ne 0) { Die 'failed to update base packaging tools' }
& $py -m pip install $wheel
if ($LASTEXITCODE -ne 0) {
Die 'installation failed. If the cryptography build is the cause, confirm OPENSSL_DIR above points at a vcpkg tree containing include\openssl\ssl.h.'
}
# ------------------------------------------------------------------- tests --
Write-Step 'Running functional checks'
$verifier = Join-Path $VenvPath 'verify_autobahn.py'
Set-Content -Path $verifier -Encoding UTF8 -Value @'
"""Post-install verification for autobahn on Windows ARM64.
Self-contained: exercises only the installed package, never the source tree.
"""
import os
import pathlib
import struct
import subprocess
import sys
import sysconfig
RESULTS = []
def check(name):
def decorate(fn):
try:
detail = fn()
RESULTS.append(True)
print(f" [PASS] {name}" + (f" -- {detail}" if detail else ""))
except Exception as exc:
RESULTS.append(False)
print(f" [FAIL] {name} -- {type(exc).__name__}: {exc}")
return fn
return decorate
@check("interpreter is native ARM64")
def _():
platform = sysconfig.get_platform()
assert platform == "win-arm64", f"got {platform}"
return platform
@check("autobahn imports")
def _():
import autobahn
# Guard against accidentally importing a source checkout instead of the wheel.
location = pathlib.Path(autobahn.__file__).parent
assert "site-packages" in str(location), f"imported from {location}"
return f"version {autobahn.__version__}"
@check("NVX native acceleration is built and active")
def _():
from autobahn.websocket import HAS_NVX, USES_NVX
assert HAS_NVX, "NVX not built into this wheel"
assert USES_NVX, "NVX built but inactive (is AUTOBAHN_USE_NVX=0 set?)"
import _nvx_xormasker
name = pathlib.Path(_nvx_xormasker.__file__).name
assert "win_arm64" in name, f"unexpected module tag: {name}"
return name
@check("shipped binaries are ARM64 machine code")
def _():
import autobahn
import _nvx_xormasker
targets = [
pathlib.Path(_nvx_xormasker.__file__),
pathlib.Path(autobahn.__file__).parent / "_flatc" / "bin" / "flatc.exe",
]
for path in targets:
with open(path, "rb") as handle:
handle.seek(0x3C)
pe_offset = struct.unpack("<I", handle.read(4))[0]
handle.seek(pe_offset)
assert handle.read(4) == b"PE\0\0", f"{path.name} is not a PE image"
machine = struct.unpack("<H", handle.read(2))[0]
# 0xAA64 == IMAGE_FILE_MACHINE_ARM64
assert machine == 0xAA64, f"{path.name} has PE machine 0x{machine:04X}, expected 0xAA64"
return f"{len(targets)} binaries verified"
@check("XOR masker round-trips")
def _():
from autobahn.websocket.xormasker import create_xor_masker
payload = bytes(range(256)) * 31
mask = b"\xde\xad\xbe\xef"
masked = create_xor_masker(mask, len(payload)).process(payload)
assert masked != payload, "masking was a no-op"
unmasked = create_xor_masker(mask, len(payload)).process(masked)
assert unmasked == payload, "round-trip mismatch"
return f"{len(payload)} bytes"
@check("UTF-8 validator accepts valid and rejects invalid input")
def _():
from autobahn.websocket.utf8validator import Utf8Validator
validator = Utf8Validator()
validator.reset()
assert validator.validate("Hello, ARM64 \u2713 \U0001f600".encode())[0], "rejected valid UTF-8"
validator.reset()
# 0xC0 0x80 is an overlong encoding of NUL and must be rejected.
assert not validator.validate(b"\xc0\x80")[0], "accepted an overlong encoding"
return "accept/reject correct"
@check("bundled flatc runs")
def _():
import autobahn
flatc = pathlib.Path(autobahn.__file__).parent / "_flatc" / "bin" / "flatc.exe"
assert flatc.is_file(), f"missing {flatc}"
out = subprocess.run([str(flatc), "--version"], capture_output=True, text=True, timeout=60)
assert out.returncode == 0, f"exit {out.returncode}: {out.stderr.strip()}"
return out.stdout.strip()
@check("cryptography works against the vcpkg OpenSSL")
def _():
from cryptography.fernet import Fernet
from cryptography.hazmat.backends.openssl.backend import backend
secret = b"windows arm64"
key = Fernet.generate_key()
assert Fernet(key).decrypt(Fernet(key).encrypt(secret)) == secret, "Fernet round-trip failed"
return backend.openssl_version_text()
@check("WebSocket echo over loopback")
def _():
import asyncio
from autobahn.asyncio.websocket import (
WebSocketClientFactory, WebSocketClientProtocol,
WebSocketServerFactory, WebSocketServerProtocol,
)
payload = b"autobahn on windows arm64 " * 64
received = {}
class Echo(WebSocketServerProtocol):
def onMessage(self, msg, isBinary):
self.sendMessage(msg, isBinary)
class Probe(WebSocketClientProtocol):
def onOpen(self):
self.sendMessage(payload, True)
def onMessage(self, msg, isBinary):
future = received["future"]
if not future.done():
future.set_result(msg)
self.sendClose()
async def exercise():
loop = asyncio.get_running_loop()
received["future"] = loop.create_future()
server_factory = WebSocketServerFactory()
server_factory.protocol = Echo
server = await loop.create_server(server_factory, "127.0.0.1", 0)
port = server.sockets[0].getsockname()[1]
client_factory = WebSocketClientFactory(f"ws://127.0.0.1:{port}")
client_factory.protocol = Probe
transport, _ = await loop.create_connection(client_factory, "127.0.0.1", port)
try:
echoed = await asyncio.wait_for(received["future"], timeout=30)
finally:
transport.close()
server.close()
await server.wait_closed()
assert echoed == payload, "echoed payload differs from what was sent"
return f"{len(payload)} bytes echoed on port {port}"
return asyncio.run(exercise())
passed = sum(RESULTS)
total = len(RESULTS)
print()
print("=" * 72)
if passed == total:
print(f"ALL CHECKS PASSED ({passed}/{total})")
else:
print(f"CHECKS FAILED ({passed}/{total} passed)")
print("=" * 72)
sys.exit(0 if passed == total else 1)
'@
& $py $verifier
$testsOk = ($LASTEXITCODE -eq 0)
# ------------------------------------------------------------------ report --
Write-Step 'Summary'
Write-Info "environment : $VenvPath"
Write-Info "activate : $VenvPath\Scripts\Activate.ps1"
Write-Info "openssl : $opensslDir"
if (-not $testsOk) {
Die 'one or more functional checks failed (see output above)'
}
Write-Host "`nautobahn is installed and working on Windows ARM64.`n" -ForegroundColor Green |
|
@ndabas — the native And yes — absolutely, welcome aboard! I'd love to get Windows ARM64 support across the rest of the A bit of framing so we pick good targets:
Process-wise, we work issue-first per repo (each change starts from a GitHub issue the PR references). Thanks again for #1937, and welcome to the WAMP crew. 🚀 |
|
Great, thanks again @oberstet! Glad to see things moving for Windows ARM64 users. If you could open tracking issues in these projects and cc me -- that would be great, I'll take a look and scope out the work needed. |
2 participants
Description
As the title says. I have added a few extra checks to make sure that the correct wheels are built, because Windows ARM64 will silently emulate AMD64, and tools like
uvcurrently default to installing the emulated Python on WoA rather than the native one.There is also a temporary step that I added specifically for Windows ARM64, to install a native OpenSSL build -- this is so that the cryptography package (which is a transitive dependency) can be built from source, as they currently have no win_arm64 wheel published. That wheel should be coming sometime soon, as the PR has already been merged. We can remove that step when that happens.
Related Issue(s)
Closes #1936.
Checklist
the style guidelines of this project
is effective or that my feature works
updated the changelog
in this PR