Problem
Every Perry build worker that cross-compiles to a target other than its
host (Linux worker → ios/macos/tvos/android/windows; the macOS worker
→ ios/tvos; etc.) currently has to build libperry_runtime.a,
libperry_stdlib.a, and libperry_ui_*.a from source for each target
on every perry update. The release tarballs ship host-only artifacts.
For the linux x86_64 host bundle:
perry-linux-x86_64.tar.gz (144 MB)
perry ← CLI
libperry_runtime.a ← LINUX HOST only
libperry_stdlib.a ← LINUX HOST only
libperry_ui_gtk4.a ← LINUX HOST only
If a worker also handles Android, iOS, macOS, tvOS, Windows — it has to
build each target's libs itself via cargo build --release -p perry-runtime -p perry-stdlib -p perry-ui-<target> --target <triple>
plus environment plumbing for the cross toolchain (CC, CXX, CFLAGS,
SDKROOT, AR, BINDGEN_EXTRA_CLANG_ARGS, and so on).
Cost we keep paying
Over the past three days a single project (PerryTS/builder linux worker
- hub) hit, recovered from, and re-hit:
aws-lc-sys jitterentropy includes macOS-only CoreServices/ CoreServices.h when compiled against the iOS / tvOS sysroot. Fix
was a systemd drop-in on the worker setting
AWS_LC_SYS_NO_JITTER_ENTROPY=1. Without it every Apple cross-target
rebuild fails non-fatally and leaves a stale .a in place, then
every user iOS/macOS build link-fails with
_perry_scene_will_connect / __perry_user_main undefined.
- Windows
.lib rebuild needs a Windows host in the perry pipeline.
Today's setup SSHes from the Linux worker into a small Azure
Windows VM, runs cargo build there, scps the resulting .libs
back. The VM disk is 30 GB and fills up mid-build at least once per
update cycle, hanging the SSH session forever (had to pkill -f perryadmin@4.207 four times in three days). Each hang stalls the
whole perry-update flow.
oboe-sys (Android C++ audio binding) needs the NDK's clang++
wrapper. cc-rs's default lookup is aarch64-linux-android-clang++
(no API version) which the NDK doesn't provide. Worker-side env fix
set CXX_aarch64_linux_android to an absolute path; without it
every Android user build with this transitive dep fails.
alsa-sys (Linux audio) needs libasound2-dev in the build
container. Not in the worker's Dockerfile before today; every Linux
native user build with this dep failed.
minimp3-sys on Windows MSVC target wants lib.exe. We pass
AR_x86_64_pc_windows_msvc=llvm-lib + CC=clang-cl now, but the
whole class of "cc-rs hardcodes a host-OS tool name" problems
recurs whenever an npm-published native dep makes a new assumption.
These are all worker-side environment-fragility problems. The perry
project itself produces the correct .as in CI for tests — that's
already proven. We're paying the cost of re-deriving the build env
across all consumers' workers, every release, forever.
Proposal
Have release CI emit per-target staticlib bundles alongside the host
bundle, one tarball per Rust target triple:
perry-linux-x86_64.tar.gz (unchanged — host)
perry-macos-x86_64.tar.gz (unchanged — host)
...
# NEW (one per cross-compile target Perry officially supports):
perry-cross-aarch64-apple-ios.tar.gz
perry-cross-aarch64-apple-darwin.tar.gz
perry-cross-aarch64-apple-tvos.tar.gz
perry-cross-aarch64-linux-android.tar.gz
perry-cross-x86_64-pc-windows-msvc.tar.gz
Each cross-target bundle contains exactly what a consumer worker needs
to LINK against without re-deriving:
perry-cross-aarch64-apple-ios.tar.gz:
libperry_runtime.a
libperry_runtime_gameloop.a (when applicable — currently swapped manually)
libperry_stdlib.a
libperry_ui_ios.a
manifest.json ← versions + sha256 of each file
Build each on the obvious CI runner (Apple targets on macos-latest with
real Xcode, Windows MSVC on windows-latest, Android on ubuntu-latest +
NDK, Linux on ubuntu-latest). All of these are runners that GitHub
Actions already provides for free, and each one is the correct host
for that toolchain — no Linux→Apple env-wrangling, no Linux→Windows
Azure-VM dance.
Worker-side simplification
The Perry-builder-linux run_perry_update function today is ~350 LOC
of cross-compile orchestration plus an SSH-to-Azure step. With shipped
prebuilts it becomes:
async fn run_perry_update(perry_binary: &str) -> Result<...> {
let version = latest_release_version().await?;
let triple = host_triple();
download_and_extract(
format!(\"perry-{triple}.tar.gz\"),
// overwrites perry CLI + host libs
).await?;
for cross in cross_targets_we_support() {
download_and_extract(
format!(\"perry-cross-{cross}.tar.gz\"),
// overwrites that target's .a's
).await?;
}
}
A 5-25-min build flow becomes a 30-second download. Every fragility
the worker hits today (AWS_LC, Azure disk, NDK lookup, alsa headers)
just stops being its problem.
Acceptance criteria
A clean Linux machine with only perry (no Rust toolchain, no Apple
SDK sysroot, no Android NDK, no Azure VM, no lib.exe) can:
perry compile src/main.ts --target ios → produces working iOS bundle
perry compile src/main.ts --target macos → produces working macOS bundle
perry compile src/main.ts --target tvos → produces working tvOS bundle
perry compile src/main.ts --target android → produces working APK
perry compile src/main.ts --target windows → produces working .exe
All by linking against the prebuilt cross-target .as shipped in the
release. Today only target #5 needs a Windows VM and only target #4
needs the NDK; with this change neither is needed.
Out of scope
- Optimizing the host bundle itself.
- npm-published native crate prebuilts (separate issue at the
bloomengine engine level — that's a different layer and a different
set of consumers).
- The Perry CLI runtime version mismatch story between a worker's
perry binary and its cross-libs — both come from the same release
tarball so they're version-locked by construction.
Concrete pain point evidence
The three-day saga that produced this issue lives in PerryTS/builder
git history and in the running session at the time, but TL;DR:
- 5 Apple cross-compile recompile cycles, each 5-25 min, each
preceded by SSH-into-Azure hang fixes;
- bloom-jump (a real user game) failed every iOS/macOS build for 48h
because the worker's libperry_runtime_ios.a was a stale ABI
against the new compiler — caused by perry-stdlib iOS rebuild
failing non-fatally inside the worker;
- 3 separate Azure-disk-full incidents during Windows .lib rebuild;
- One full hub OOM crash whose root cause was unrelated but whose
recovery was complicated by the worker being mid-cross-compile.
None of this would happen if the release CI shipped the libs.
Problem
Every Perry build worker that cross-compiles to a target other than its
host (Linux worker → ios/macos/tvos/android/windows; the macOS worker
→ ios/tvos; etc.) currently has to build
libperry_runtime.a,libperry_stdlib.a, andlibperry_ui_*.afrom source for each targeton every
perry update. The release tarballs ship host-only artifacts.For the linux x86_64 host bundle:
If a worker also handles Android, iOS, macOS, tvOS, Windows — it has to
build each target's libs itself via
cargo build --release -p perry-runtime -p perry-stdlib -p perry-ui-<target> --target <triple>plus environment plumbing for the cross toolchain (CC, CXX, CFLAGS,
SDKROOT, AR, BINDGEN_EXTRA_CLANG_ARGS, and so on).
Cost we keep paying
Over the past three days a single project (PerryTS/builder linux worker
aws-lc-sysjitterentropy includes macOS-onlyCoreServices/ CoreServices.hwhen compiled against the iOS / tvOS sysroot. Fixwas a systemd drop-in on the worker setting
AWS_LC_SYS_NO_JITTER_ENTROPY=1. Without it every Apple cross-targetrebuild fails non-fatally and leaves a stale
.ain place, thenevery user iOS/macOS build link-fails with
_perry_scene_will_connect/__perry_user_mainundefined..librebuild needs a Windows host in the perry pipeline.Today's setup SSHes from the Linux worker into a small Azure
Windows VM, runs
cargo buildthere, scps the resulting.libsback. The VM disk is 30 GB and fills up mid-build at least once per
update cycle, hanging the SSH session forever (had to
pkill -f perryadmin@4.207four times in three days). Each hang stalls thewhole perry-update flow.
oboe-sys(Android C++ audio binding) needs the NDK's clang++wrapper. cc-rs's default lookup is
aarch64-linux-android-clang++(no API version) which the NDK doesn't provide. Worker-side env fix
set
CXX_aarch64_linux_androidto an absolute path; without itevery Android user build with this transitive dep fails.
alsa-sys(Linux audio) needslibasound2-devin the buildcontainer. Not in the worker's Dockerfile before today; every Linux
native user build with this dep failed.
minimp3-syson Windows MSVC target wantslib.exe. We passAR_x86_64_pc_windows_msvc=llvm-lib+CC=clang-clnow, but thewhole class of "cc-rs hardcodes a host-OS tool name" problems
recurs whenever an npm-published native dep makes a new assumption.
These are all worker-side environment-fragility problems. The perry
project itself produces the correct
.as in CI for tests — that'salready proven. We're paying the cost of re-deriving the build env
across all consumers' workers, every release, forever.
Proposal
Have release CI emit per-target staticlib bundles alongside the host
bundle, one tarball per Rust target triple:
Each cross-target bundle contains exactly what a consumer worker needs
to LINK against without re-deriving:
Build each on the obvious CI runner (Apple targets on macos-latest with
real Xcode, Windows MSVC on windows-latest, Android on ubuntu-latest +
NDK, Linux on ubuntu-latest). All of these are runners that GitHub
Actions already provides for free, and each one is the correct host
for that toolchain — no Linux→Apple env-wrangling, no Linux→Windows
Azure-VM dance.
Worker-side simplification
The Perry-builder-linux
run_perry_updatefunction today is ~350 LOCof cross-compile orchestration plus an SSH-to-Azure step. With shipped
prebuilts it becomes:
A 5-25-min build flow becomes a 30-second download. Every fragility
the worker hits today (AWS_LC, Azure disk, NDK lookup, alsa headers)
just stops being its problem.
Acceptance criteria
A clean Linux machine with only
perry(no Rust toolchain, no AppleSDK sysroot, no Android NDK, no Azure VM, no
lib.exe) can:perry compile src/main.ts --target ios→ produces working iOS bundleperry compile src/main.ts --target macos→ produces working macOS bundleperry compile src/main.ts --target tvos→ produces working tvOS bundleperry compile src/main.ts --target android→ produces working APKperry compile src/main.ts --target windows→ produces working .exeAll by linking against the prebuilt cross-target
.as shipped in therelease. Today only target #5 needs a Windows VM and only target #4
needs the NDK; with this change neither is needed.
Out of scope
bloomengine engine level — that's a different layer and a different
set of consumers).
perry binary and its cross-libs — both come from the same release
tarball so they're version-locked by construction.
Concrete pain point evidence
The three-day saga that produced this issue lives in PerryTS/builder
git history and in the running session at the time, but TL;DR:
preceded by SSH-into-Azure hang fixes;
because the worker's
libperry_runtime_ios.awas a stale ABIagainst the new compiler — caused by perry-stdlib iOS rebuild
failing non-fatally inside the worker;
recovery was complicated by the worker being mid-cross-compile.
None of this would happen if the release CI shipped the libs.