perf(ci): cut cloud parity time — public static site + sharded BrowserStack (CDP)#302
perf(ci): cut cloud parity time — public static site + sharded BrowserStack (CDP)#302RaananW wants to merge 26 commits into
Conversation
…ck tunnel
Phase 2 of the CI BrowserStack overhaul. Parity (Cloud) is the CI
wall-clock bottleneck and is bimodal (~47 min vs ~114 min for the same
~198 scenes). The dominant source of variance is the always-on
BrowserStack Local tunnel that fronts a single serial session: one slow
tunnel taxes every one of the 198 serially-run scenes.
This removes the tunnel from the cloud parity path by serving a
self-contained static parity site from a public, build-isolated URL and
pointing the remote browser straight at it.
- tests/lite/parity: navigate with baseURL-relative paths ("sceneN.html")
instead of absolute ("/sceneN.html") so the same specs resolve under a
path-prefixed public host or localhost.
- playwright.config.ts + parity-cloud config: add use.baseURL. When
PARITY_BASE_URL is set, parity-cloud uses it as baseURL and drops the
local webServer; otherwise it keeps the local dev-server fallback.
- browserstack.yml: browserstackLocal is now ${BROWSERSTACK_LOCAL};
run-browserstack.ts disables the tunnel when PARITY_BASE_URL is set
(assets already come from public CDNs), else defaults it on.
- azure-pipelines.yml ParityCloud: build + upload a build-isolated static
parity site and run parity against its public URL (no tunnel). Upload
template gains an optional postComment flag (no PR comment for parity).
- build-lab-site.ts: fix latent double-prefix bug — when the deploy base
path itself starts with a reserved prefix (e.g. /lite/<build>/...),
Vite-emitted asset URLs were prefixed a second time. The rewrite now
skips URLs already under the base path.
- TESTING.md: document the dual-mode (public site vs local tunnel) parity
serving.
Local fallback is preserved, so this is non-breaking.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…a CDP Phase 1 of the CI BrowserStack overhaul. The cloud parity job ran all ~198 scenes serially on a single BrowserStack session (the SDK with parallelsPerPlatform: 1), which is the CI wall-clock bottleneck and the source of its 47-min↔114-min bimodal variance. Migrate parity to a direct CDP connection and shard it across parallel cloud sessions: - playwright.parity-cloud.config.ts: connect via Playwright's connectOptions.wsEndpoint (wss://cdp.browserstack.com/playwright) with capabilities built in-config — no browserstack-node-sdk, no browserstack.yml, no Local tunnel. fullyParallel + workers=CIWORKERS so each worker is its own cloud session. Keeps the public-site baseURL and a local-Chrome fallback. trace is forced off (the wsEndpoint embeds the access key). - scripts/browserstack-wait.sh: poll the BrowserStack plan API, grab up to BSTACK_SESSIONS_REQUIRED sessions (fall back to fewer when busy), export CIWORKERS, and retry on queue-size-exceeded. - scripts/redact-secrets.ts + pipeline step: strip BrowserStack creds from test-results before the report/JUnit are published, in case a connection error echoes the wsEndpoint. - azure-pipelines.yml ParityCloud: run via the wait script with BSTACK_SESSIONS_REQUIRED=5; redact artifacts before publishing. - package.json: test:parity-cloud now runs Playwright directly. - browserstack.yml / run-browserstack.ts: reverted to master — they are now used only by the perf job, which still uses the SDK + tunnel. - TESTING.md: document the parity (CDP/sharded) vs perf (SDK/tunnel) split. Perf and local dev are unchanged, so this is non-breaking. Expected effect: parity slow bucket ~114 min -> ~23 min at 5 sessions, with the variance largely collapsed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR aims to cut CI wall-clock time for Parity (Cloud) by removing the BrowserStack Local tunnel dependency and parallelizing parity execution across multiple BrowserStack sessions via direct Playwright CDP. It also makes parity/bundle-size navigation baseURL-relative so the same specs can run against a path-prefixed, build-isolated public static site in CI (and still work locally).
Changes:
- Switch Parity (Cloud) to Playwright
connectOptions.wsEndpoint(no BrowserStack SDK /browserstack.yml) and shard across parallel sessions controlled viaCIWORKERS. - Build + upload a public static parity site (
pnpm build:lab-site) and run cloud parity against it viaPARITY_BASE_URL(no Local tunnel). - Update parity and bundle-size tests to use baseURL-relative
page.goto(...), add local baseURL toplaywright.config.ts, and add CI artifact credential redaction.
Reviewed changes
Copilot reviewed 209 out of 209 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| azure-pipelines.yml | ParityCloud job now builds/uploads a static site, runs sharded CDP parity, and redacts secrets before publishing artifacts. |
| config/playwright.parity-cloud.config.ts | Implements CDP BrowserStack connection, worker sharding via CIWORKERS, and public PARITY_BASE_URL page sourcing. |
| config/templates/upload-static-site.yml | Adds optional postComment parameter to suppress PR comments for uploaded sites. |
| package.json | Runs parity-cloud directly via Playwright config (no SDK wrapper). |
| playwright.config.ts | Defines use.baseURL for local/CI runs so scene specs can navigate with relative URLs. |
| scripts/browserstack-wait.sh | Waits for free BrowserStack sessions, sets CIWORKERS, retries on queue/capacity contention. |
| scripts/build-lab-site.ts | Fixes double-prefixing when rewriting root-relative URLs under a base path that begins with a reserved prefix. |
| scripts/redact-secrets.ts | New script to strip BrowserStack credentials from published test artifacts (HTML/JUnit/etc). |
| TESTING.md | Updates documentation to reflect the new Parity (Cloud) CDP + sharded model and static-site sourcing. |
| tests/lite/parity/bundle-size.spec.ts | Bundle-size pages now navigated via baseURL-relative URLs. |
| tests/lite/parity/compare-utils.ts | captureGolden BJS reference navigation converted to baseURL-relative. |
| tests/lite/parity/scenes/scene1-boombox.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene2-sphere.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene3-fog.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene4-shadows.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene5-alien.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene6-pbr-sphere.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene7-chibirex.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene8-glass-sphere.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene9-sponza.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene10-pbr-rough.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene11-shark.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene12-shader-balls.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene13-pbr-spheres.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene14-flight-helmet.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene15-spotlights.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene16-thin-instances.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene16-culling.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene17-pbr-std-thin-instances.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene18-spotlight-shadows.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene19-clearcoat.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene20-emissive-grid.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene21-sheen-cloth.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene22-pbr-shadows.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene23-anisotropy.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene24-hillvalley.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene25-ktx-texture.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene26-pbr-subsurface.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene27-material-variants.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene28-clearcoat-gltf.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene29-sheen-cloth-gltf.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene30-volume-testing.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene31-emissive-strength.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene32-unlit.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene33-lights-punctual.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene34-node-visibility.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene35-gltf-instancing.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene36-basis-texture.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene37-sheen-sofa.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene38-procedural-builders.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene39-animation-pointer-waterfall.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene40-physics.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene41-physics-shapes.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene42-physics-clone.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene43-parametric-proximity.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene44-physics-sleeping-towers.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene45-physics-filtering.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene46-constraint-viewer.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene47-physics-heightfield.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene48-physics-center-of-mass.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene49-physics-shape-queries.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene50-sprite-grid.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene51-sprite-grid.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene52-hud-on-3d.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene53-depth-hosted-sprites.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene54-facing-billboards.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene55-billboard-sorting.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene56-axis-locked-billboards.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene57-cutout-billboards.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene58-sprite-animation.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene59-billboard-animation.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene60-nme-flat.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene61-nme-normal.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene62-nme-texture.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene63-nme-light.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene64-nme-morph.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene65-nme-shadow.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene66-nme-big.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene67-nme-pbr-core.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene68-nme-pbr-clearcoat.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene69-nme-pbr-sheen.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene70-nme-pbr-aniso.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene71-nme-pbr-subsurface.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene72-nme-pbr-full.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene73-wheel-nme-viewport.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene74-effect-renderer.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene75-effect-rtt-sphere.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene76-effect-texture.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene77-nme-compat.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene78-nme-math.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene79-nme-modes.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene80-nme-color.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene81-nme-uv-projection.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene82-nme-noise.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene83-nme-normals.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene84-nme-fragment-screen.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene85-nme-matrix.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene86-nme-scene-state.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene87-nme-iridescence-image.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene88-nme-loop.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene89-nme-storage.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene90-csg.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene91-csg2.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene92-sprite-customshader-params.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene93-sprite-customshader-palette.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene94-billboard-customshader-params.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene95-billboard-customshader-palette.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene96-sprite-uvoffset-parallax.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene97-sprite-multiply-blend.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene98-billboard-additive-blend.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene99-bone-control.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene100-physics-collision.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene101-physics-trigger.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene102-physics-raycast.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene103-thin-instance-raycast.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene104-character-controller.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene105-moving-platform.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene106-prestep-motion-types.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene110-rtt-override.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene111-light-selection.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene112-khr-texture-basisu.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene113-picking-precision.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene114-morph-skeleton-picking.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene115-alien-picking-frame100.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene116-shadow-depth-materials.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene120-gaussian-splatting.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene121-gs-update-data.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene122-gs-sog.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene123-gs-spz.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene124-gs-compressed-ply.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene125-gs-bake-transform.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene126-gs-material-plugin.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene127-gs-depth-rendering.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene128-gs-depth-rendering-alpha.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene129-gs-gpu-picking.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene140-nme-pcf-alpha-discard-shadows.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene141-esm-material-casters.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene142-black-and-white-post-process.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene143-pipelined-post-processes.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene144-bloom-post-process.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene145-geometry-renderer.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene146-pbr-geometry-renderer.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene147-circle-of-confusion.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene148-depth-of-field.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene149-node-geometry-renderer.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene150-manual-x-slide.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene151-manual-transform.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene152-gltf-manager.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene153-autonomous-manager.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene154-step-time-animation.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene155-manual-weighted-blend.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene156-manual-cross-fade.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene157-gltf-weighted-blend.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene158-additive-animation-blend.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene159-shader-material-basic.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene160-shader-material-texture.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene161-shader-material-uniform.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene162-shader-material-defines.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene163-shader-material-alpha.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene164-device-lost-recovery.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene165-shader-material-thin-instances.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene170-navigation-basic.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene171-navigation-crowd-path.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene172-navigation-obstacles.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene173-navigation-dynamic-obstacles.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene174-navigation-offmesh.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene175-navigation-raycast.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene176-mosquito-amber.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene177-pbr-iridescence.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene178-gltf-iridescence-abalone.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene179-clustered-sponza-lights.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene200-high-precision-jitter-hpm-off.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene201-high-precision-jitter-hpm-on.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene202-floating-origin-point-light.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene203-floating-origin-spot-light.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene204-floating-origin-thin-instances.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene205-floating-origin-facing-billboards.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene206-floating-origin-cutout-billboards.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene207-floating-origin-directional-shadows.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene209-floating-origin-physics.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene210-xmp-metadata-cube.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene211-meshopt-brainstem.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene212-gltf-dispersion.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene213-gridmaterial.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene214-cascaded-shadows.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene215-cascaded-shadows-pbr.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene216-pbr-fog.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene217-material-plugin.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene218-vat.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene219-vat-instanced.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene221-pointer-drags.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene222-composite-gizmos.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene223-camera-light-gizmos.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene224-bounding-box-gizmo.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene225-geospatial-camera.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene229-triangle-without-indices.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene240-animated-triangle.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene241-animation-pointer-uvs.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene242-emissive-fireflies.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene243-morph-stress-test.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene244-pot-of-coals-animation-pointer.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene245-recursive-skeletons.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene246-simple-skin.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene247-teapots-galore.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene248-texture-settings-test.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene249-vertex-color-alpha-clip-test.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene251-animation-mask.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene252-standard-material-morph.spec.ts | Scene spec navigations converted to baseURL-relative. |
| tests/lite/parity/scenes/scene253-animate-all-the-things.spec.ts | Scene spec navigations converted to baseURL-relative. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Limit the parity job to at most 2 parallel BrowserStack sessions (min 1) so a single PR no longer claims the whole 5-session plan. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…wserstack-analysis # Conflicts: # package.json # tests/lite/parity/compare-utils.ts
- compare-core captureGolden: manually-created BrowserStack ref contexts do not inherit the project use.baseURL, so baseURL-relative ref URLs failed to resolve under the path-prefixed public host. Forward the project baseURL to the new context. - parity-cloud config: error before claiming a BrowserStack session when credentials are present but PARITY_BASE_URL is unset (the remote browser cannot reach localhost without a tunnel). Addresses Copilot review feedback on PR #302. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…llers The vitest unit test exercises captureGolden with a mocked browser outside the Playwright runner, where test.info() throws. Wrap the baseURL lookup in try/catch so it resolves the project baseURL under Playwright and falls back to undefined for vitest unit tests / standalone recapture scripts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Lab - Static SiteBuild 20260625.15 - merge @ c223328 |
The parity static-site build ran rewriteRootRelativeUrls() over Vite's minified Babylon.js chunks, matching after (, =, : and whitespace. That mangled regex literals such as .replace(/gl_FragColor/g, ...) into .replace(/lite/<build>/parity-lab/gl_FragColor/g, ...) (invalid regex flags -> chunk crash) and glTF JSON pointers like m.Get(\\\/scene\\\). Reference pages then failed to load, so every live-captured golden scene timed out in cloud parity. For .js/.json the rewriter now only matches inside quoted string literals and only rewrites tokens that look like real asset URLs (file extension or sub-path), leaving regex literals and JSON pointers untouched while still basing genuine runtime URLs (scene-config.json, reference/, textures/, HavokPhysics.wasm, etc.). CSS/HTML keep the original broad behaviour. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Lab - Static SiteBuild 20260625.20 - merge @ cbc28ed |
…emplate Every CI job re-ran corepack + 'pnpm install --frozen-lockfile' from scratch, and two jobs re-downloaded Playwright browsers. Extract the bootstrap into a reusable config/templates/install-deps.yml step template that adds a cross-run pnpm store cache (keyed on pnpm-lock.yaml) and an optional Playwright browser cache, then reference it from all eight jobs. Dependency installs now hit the cache on the common unchanged-lockfile path, and the duplicated corepack/install blocks collapse to a single template. Note: build:bundle-scenes (3 jobs) and build:lab-site (2 jobs) are intentionally not cached — their output changes on every commit, so a cache would never hit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bundle Size ChangesDecreases
Sizes rounded to nearest KB. Run |
Lab - Static SiteBuild 20260625.27 - merge @ e752390 |
The plan now allows 3 parallel sessions, so let parity shard across up to 3 (was 2). Still falls back to a single session when the plan is busy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Lab - Static SiteBuild 20260625.30 - merge @ b077222 |
The .gitignore was silently ignoring every reference/lite/**/babylon-ref-golden.png
(only 59 of 204 scenes were committed via git add -f). Scenes without a committed
golden fall into captureGolden()'s live-capture path, booting a full Babylon.js
reference page on every CI run (~10x slower per scene, network-flaky) — which
contradicts GUIDANCE §2c ('no parity test may open a BJS reference page at
runtime').
- .gitignore: keep transient babylon-ref-*.png ignored but explicitly allow
babylon-ref-golden.png, so new goldens commit without git add -f.
- TESTING.md: rewrite the Golden References section (performance rationale,
macOS-capture requirement, recapture commands) and add an 'Adding a new scene'
checklist; fix stale spec count and page name.
- GUIDANCE.md: make the new-scene golden step explicit (capture on macOS, commit
in the same PR, goldens are tracked).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add an opt-in strict mode to captureGolden that throws when a scene's committed golden is missing, instead of silently live-capturing the Babylon.js reference page at runtime (slow, ~10x per scene, forbidden by GUIDANCE 2c). Wire PARITY_REQUIRE_GOLDEN=false into the cloud parity job with a flip-to-enable comment, and document the env var in TESTING.md. Default OFF so the current run still passes while the 144 missing goldens are captured on macOS and committed; flip to true afterwards. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…l goldens The golden capture swept up 69 goldens into a legacy top-level reference/<slug>/ tree that no test references (lite specs read reference/lite/<slug>, gl specs read reference/gl/<slug>). They leaked in because the .gitignore golden negation un-ignores goldens in every tree, so git add grabbed old previously-ignored captures off disk. - Move scene40-physics golden from reference/scene40-physics/ to reference/lite/scene40-physics/ (it had landed only at the top level, leaving the lite spec without a golden). - Delete the remaining 68 orphaned top-level goldens. Result: 204/204 lite parity specs have a committed golden, no orphans. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Lab - Static SiteBuild 20260625.36 - merge @ 003d29a |
Blanket RECAPTURE_GOLDEN regenerated all 203 goldens, rewriting 53 that were already committed and valid. The changes are pure render noise (51 of 53 below MAD 0.5, 39 below 0.1; the two largest ~1.0, all within each scene's maxMad threshold) -- no visual change, just churn on large binary PNGs. Restore those 53 to their original committed bytes; keep only the 144 genuinely-new goldens. Going forward the harness only overwrites an existing golden when RECAPTURE_GOLDEN is set, so normal runs won't churn. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document that WebGPU recapture is non-deterministic, so a blanket RECAPTURE_GOLDEN run rewrites every golden even with no visual change. Contributors should commit only new scenes or genuine visual updates and git checkout -- the noise-only diffs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Lab - Static SiteBuild 20260625.39 - merge @ 6ecccb5 |
📋 parity — Test ReportBuild 20260625.39 · merge @ 6ecccb5 |
13 cloud parity tests failed because asset/page URLs bypassed the new per-build base path on the public static site and 404'd at the domain root: - scene254/255/257/258/259/260: page.goto used a leading-slash path (/sceneN.html) that resolves to the domain root, ignoring the base path. Made the goto paths relative so they resolve against PARITY_BASE_URL. - scene170-175 (recast) and scene211 (meshopt): the lab-site URL rewriter allowlist (ROOT_RELATIVE_PREFIXES) rebases HavokPhysics.wasm and draco but not recast-navigation.wasm or meshopt_decoder.js, so their wasm/JS 404'd and init hung until the 120s timeout. Added both to the allowlist. - scene211 also composes the meshopt decoder URL at runtime (default base '/'), so the allowlist alone can't catch it; set the meshopt base relative to the page URL so it works under any base path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📋 parity — Test ReportBuild 20260625.43 · merge @ 6d1b5aa |
Parity scenes referenced root-absolute asset URLs (NME texture maps baked into compressed snippet data, and the Draco/meshopt decoder scripts) that resolve against the site root. Under the per-build base path used by the cloud parity static site they 404, so scene72 fell back to a plain sphere, scene66/140 threw on the missing textures and never became ready, and scene30 hung waiting on /draco_decoder.js. Add lab/lite/src/shared/asset-url.ts which rebases such URLs at runtime against document.baseURI (the proven scene211 precedent) instead of the build-time rewriter, which cannot see URLs hidden in compressed blobs or constructed at runtime. Wire it into the NME texture loads (scene72/66/140) and the Draco/meshopt decoder bases (scene30). Regenerate the affected per-scene bundle manifests (scene211 also picks up its earlier meshopt edit). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📋 parity — Test ReportBuild 20260626.9 · merge @ fd32ad2 |
Bundle Size ChangesIncreases
Sizes rounded to nearest KB. Run |
…wserstack-analysis # Conflicts: # lab/public/bundle/manifest/scene140.json # lab/public/bundle/manifest/scene211.json
Bundle Size ChangesIncreases
Sizes rounded to nearest KB. Run |
📋 parity — Test ReportBuild 20260627.6 · merge @ f51d6b6 |
…wserstack-analysis # Conflicts: # lab/public/bundle/manifest/scene211.json # lab/public/bundle/manifest/scene30.json
Frame-graph changes from master (depth-resolve/transmission) shifted per-scene bundle sizes. Regenerated scene30/scene211 (merge-conflicted) plus scene66/scene140 so the committed manifest matches a fresh build and passes validate:bundle-manifest. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
scene261 (temporal AA) merged from master used an absolute-path goto
("/scene261.html"), which breaks under the build-number-prefixed public
parity base URL (PARITY_BASE_URL). Drop the leading slash to match the
rest of the suite so the remote BrowserStack browser resolves it against
the deployed static site path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📋 parity — Test ReportBuild 20260628.8 · merge @ 807438f |
📋 perf — Test ReportBuild 20260628.8 · merge @ 807438f |
Bundle Size ChangesIncreases
Sizes rounded to nearest KB. Run |
… lean The parity static-site work wired configureParityDecoderBases() into scene30 (and an inline equivalent into scene211) that eagerly import()-ed the Draco and meshopt decoder wrappers solely to call their base-URL setters. That static import pulled both decoders into the scene bundle — defeating their zero-byte lazy-load design and pushing scene30 (+draco+meshopt) and scene211 (+meshopt) over their committed size ceilings. scene30 doesn't even use meshopt. Make draco-decode.ts and meshopt-decode.ts resolve their base URL lazily from a new __babylonLiteDecoderBase__ global (explicit setDraco/MeshoptBaseUrl() still take precedence; default stays "/"). configureParityDecoderBases() now just sets that global synchronously — no import — so the decoders are only pulled in when a scene actually loads a compressed asset (the glTF feature dynamic-imports them on demand and they pick up the base). scene30 104.3 -> 103.4 KB (ceiling 103.5), scene211 90.5 -> 90.3 KB (ceiling 90.4): both back under ceiling with no ceiling change. Backward compatible — demos using the explicit setters are unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bundle Size ChangesIncreases
Sizes rounded to nearest KB. Run |
📋 parity — Test ReportBuild 20260628.10 · merge @ ac138af |
…wserstack-analysis # Conflicts: # lab/public/bundle/manifest/scene140.json # lab/public/bundle/manifest/scene211.json # lab/public/bundle/manifest/scene30.json # lab/public/bundle/manifest/scene72.json
… under tree-shaking The glTF primitive feature and KHR_animation_pointer feature installed the PBR primitive-state resolver and the non-Float32 sampler converter via bare side-effect imports (`import "...";`). Under the engine's `"sideEffects": false`, Rollup drops those imports from the per-scene production bundles, so: - non-triangle topologies / negative-winding meshes silently fell back to plain triangle-list (scene257 negative-node-scale, scene260 triangle-strip rendered wrong: parity MAD 2.33 / 26.37 vs 0.1 limit), and - normalized / non-Float32 animation samplers reverted to the Float32 fast-path reinterpret. Wrap each install in an idempotent exported initializer and call it through a used binding from the feature module, so the reference survives tree-shaking regardless of the `sideEffects` hint. Triangle-list scenes still never load these modules, so their bundles stay byte-identical. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Bundle Size ChangesIncreases
Sizes rounded to nearest KB. Run |
📋 parity — Test ReportBuild 20260701.9 · merge @ cacdf4a |
Why
The CI wall-clock is dominated by the Parity (Cloud) job, which runs all ~198 parity scenes serially on a single BrowserStack session behind an always-on Local tunnel. Real Azure DevOps run data (pipeline def #44, last 40 runs) shows it is bimodal for the same scene set: 23 runs averaged 47 min, 17 runs averaged 114 min (max 123.5; some canceled near the 2 h timeout). One slow session taxes all 198 serial scenes, and there's no parallelism to absorb it.
This PR addresses both root causes — the tunnel and the serial single session — matching how Babylon.js main uses BrowserStack (direct CDP, parallelized), while keeping the perf job and local dev unchanged.
What changed
Phase 2 — drop the Local tunnel (
59132d0c)pnpm build:lab-site→ upload), and the remote browser loads them directly viaPARITY_BASE_URL. No tunnel."sceneN.html"), withuse.baseURLadded to both Playwright configs so the same specs work locally and against a path-prefixed public host.build-lab-site.ts(when the deploy base path itself starts with a reserved prefix likelite, Vite-emitted asset URLs were prefixed twice — also affected the existing Bundle Size deploy).Phase 1 — shard across parallel sessions via CDP (
780ac474)connectOptions.wsEndpoint(wss://cdp.browserstack.com/playwright) — nobrowserstack-node-sdk, nobrowserstack.yml. Each Playwright worker is its own cloud session, so the ~198 specs shard across N parallel browsers.scripts/browserstack-wait.shgrabs up toBSTACK_SESSIONS_REQUIRED(5) sessions from the plan, falls back to fewer when busy, exportsCIWORKERS, and retries on queue-size-exceeded.scripts/redact-secrets.ts+ a pipeline step strip BrowserStack creds fromtest-resultsbefore the report/JUnit are published (the CDP wsEndpoint embeds the access key);traceis forced off.browserstack.yml/run-browserstack.tsreverted to master — they are now used only by the perf job, which keeps the SDK + tunnel.Expected impact
Verification
scene1.htmlunder the prefixed base path → 0 failed requests (fully self-contained; the only error was local "WebGPU adapter not available", which BrowserStack provides).CIWORKERS,fullyParallel, correct caps incl.playwrightVersion,webServerdropped when public URL set); both fallback paths verified.format:checkclean; eslint 0 errors; all YAML valid;browserstack-wait.shpassesbash -n.Notes / follow-ups
srcchanges), so the parity/bundle suites are not required by the repo guardrails; perf is never run by agents.build:bundle-scenesstep in the ParityCloud job, and consider moving the perf job off the SDK/tunnel too.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com