[Performance] Load each .untold of a pack once

[miogds]

Summary

Stage 1b of docs/proposals/LargeSceneRendering.md. A .untoldpack can place one .untold many times (since #1288 the exporter writes a repeated model once), but the pack loader still read, decoded and built GPU buffers for every placement. Now each distinct file is loaded once per pack load, image textures shared between models are decoded once, and the memory budget counts what placements share once.

BIM site pack (10,314 placements, 4,583 files)	Before	After
Load time	118.3 s	20.0 s
GPU memory (MTLDevice.currentAllocatedSize)	57.7 GB	3.8 GB
Reported by MemoryBudgetManager	52.2 GB	3.8 GB

Measured headless on an M-series Mac with the same scratch benchmark on both sides, on upstream develop with #1289, which this pack needs because it creates 24,162 entities.

Changes

• UntoldBuildCache (owned by PackLoadDispatcher for the duration of one pack load):
  • The first placement of a file parses it and runs prebuildNodeMeshes.
  • Placements that arrive while it is building suspend until it is done rather than building again. This is single-flight with continuations, so no thread is blocked; the dispatcher's existing concern about blocking the cooperative pool is untouched.
  • Every placement registers its own copy of the built Mesh values.
  • A build that fails is tried once more. A file that fails twice is remembered, so its other placements fall back without reading it again.
  • The table of named states lives inside an OSAllocatedUnfairLock, held for a few instructions at a time and never while a file is built or a waiting caller is resumed.
• setEntityMeshAsync forwards to loadEntityMeshAsync, which takes the cache. Callers outside a pack get nil, i.e. today's behaviour. The cache is used only for whole-asset .immediate loads, which is what pack entries are; the test that decides it says so.
• LoadedTextureCache: image textures loaded by Material(runtimeMaterial:) are cached by file and colour space, with weak values. A texture lives only while some material holds it, so the cache never keeps memory alive on its own. It is single-flight, as NativeTextureLoader is: a thread that asks for an image another one is decoding waits for it. .utex textures already had NativeTextureLoader.sharedCache.
• MemoryBudgetManager: registerMesh(entityId:meshes:) registers the buffers and textures an entity holds, and the totals count each one once while any registered entity holds it (weak references, by identity).
  • getMemorySize(for:) and the eviction candidates report what evicting an entity would free; a shared buffer goes with its last holder.
  • Sizes registered as numbers, as the streaming paths do, count as before.

Why sharing is safe

• Mesh, SubMesh and Material are structs. Their GPU data (MTKMesh, buffers, textures, MorphTargetSet) are references.
• Every mutation goes through the entity's own array: updateMaterial, assignRuntimeSkins, LOD and streaming swaps. Copy-on-write therefore keeps material edits and skins per entity.
• Mesh.cleanUp() only affects its own copy, and GPU memory is released by ARC when the last copy goes.
• Loaded textures are never written in place: material edits and texture streaming replace a material's texture rather than change it.
• The deformation system's scratch buffers keyed by ObjectIdentifier(metalKitMesh) are used serially within one compute encoder.

Verification

• UntoldBuildCacheTests (6): 32 concurrent requests for one file build it once and all get the same build; two different files build at the same time; another path to the same file is the same build; a build that fails once is tried again; a file that fails twice is remembered; callers waiting for a failing build all fall back.
• MemoryBudgetManagerTests (+11): a buffer that 500 entities hold is counted once; it leaves the totals with its last holder; an entity's own buffers add to the shared ones; evicting an entity frees only what no other holds; eviction to a target counts a shared buffer with its last holder; a number registered for an entity replaces its allocations; an address handed out again is a new allocation.
• UntoldPackRenderTests (+6):
  • two placements of one file share the same MTKMesh while a separate file with the same content gets its own, and each keeps its transform;
  • a roughness edit on one placement leaves the other unchanged;
  • three placements of one file are in the memory budget once, and leave it with the last placement;
  • the texture cache returns the same texture while it is held and nothing after it is released; eight threads asking for one image decode it once; a load that fails is tried again.
• swift test --filter UntoldEngineTests: 1531 run. The only failure is ExternalRenderExtensionPackageTests, which fails locally whenever the checkout folder is not named UntoldEngine, unrelated.
• The render suite (1133 tests) and make testrenderer-async (31): no failure beyond the 18 reference-image comparisons that need Python packages this machine does not have.
• Zero warnings in the strict-concurrency build; no new SwiftFormat findings in the changed files.

Limits

• A texture size that texture streaming updates (updateTextureSizeBytes) is recorded per entity, as before, even when several entities end up with the same streamed texture.
• Counting by identity also counts once what a single entity used twice (an image in two slots of a material) and a .utex that several entities share through NativeTextureLoader's cache, so texture totals can be lower than before for existing scenes.

[untoldengine]

Nice work on this — the single-flight build cache and shared texture cache are a clean way to kill the redundant-parse/decode cost for repeated pack placements, and the tests covering mesh-sharing + material-isolation + texture lifetime are a great touch. Left a few inline notes below, roughly in order of how much I'd want eyes on them before merge. Happy to pair on any of these if useful!

[untoldengine]

Heads up on a side effect of sharing prebuiltMeshes here: registerRenderComponent (further down in this file) calls MemoryBudgetManager.shared.registerMesh(entityId:meshes:) for every entity, and that computes size from the real MTLBuffer.length (Mesh+GPUMemory.swift). Since N placements now share one buffer, each placement's registerMesh call charges the full buffer size again — so a model placed 500 times would report ~500x its actual GPU footprint to the budget manager.

Before this PR that accounting was correct (each placement really did own its own buffer), so this looks like a new regression rather than a pre-existing issue. Might be worth either registering the shared bytes once per UntoldBuild (e.g. only on the first placement, or tracked keyed by the build rather than per entity) or having MemoryBudgetManager dedupe by buffer identity. Could otherwise cause phantom memory pressure that evicts unrelated streaming entities on big packs.

[miogds]

You're right, and it was worse than a wrong number. On the BIM pack this PR was written for, the budget reported 52.2 GB against 4.8 GB really allocated, and sat in its eviction state on a Mac with a 26 GB geometry budget. In that state geometry streaming holds back its loads and StreamingRegionManager destroys the least recently used entities, so the note in the description that this could wait for a follow-up was wrong.

Fixed in f114811, by identity as you suggest: registerMesh(entityId:meshes:) registers the buffers and textures the entity holds, and the totals count each one once while any registered entity holds it. The ledger knows them through weak references, so it never keeps GPU memory alive and an address that is handed out again is not taken for the buffer that was there before. The same pack now reports 3.77 GB for 3.83 GB allocated.

Two things follow from it:

• getMemorySize(for:) and the eviction candidates now say what evicting an entity would free. A placement that shares its buffers frees nothing until it is the last one, and getEvictionCandidatesToTarget() counts a shared buffer with its last holder.
• What one entity used twice was counted twice (an image in two slots of a material), and a .utex that several entities got from NativeTextureLoader's cache was counted for each of them. Both are counted once now, so texture totals can come out lower than before for existing scenes.

Sizes registered as numbers (the streaming paths, updateTextureSizeBytes) count as they did.

[untoldengine]

Nice idea caching decoded textures, but this check-then-store isn't single-flight the way UntoldBuildCache below is — there's no "in progress" marker, just the lookup and the eventual store(). Given PackLoadDispatcher fires up to 8 concurrent loads, two different models that share the same pack texture can both miss the cache before either finishes decoding, and you'd end up decoding/uploading the same PNG twice (the second store() just overwrites the first). Not a correctness bug since each caller still gets a valid texture, but it does undercut the "decode it once" goal in the doc comment above, under exactly the concurrency this PR introduces. Might be worth mirroring the in-flight tracking you already built for UntoldBuildCache/NativeTextureLoader.

[miogds]

Agreed, and it cost memory as well as time: the duplicates stayed alive for as long as their materials did. Fixed in af6b706. LoadedTextureCache is now single-flight the way NativeTextureLoader is, with a condition and the set of keys being loaded, since materials are built synchronously: a thread that asks for an image another one is decoding waits and takes its texture, and a load that fails is not remembered. On the BIM pack that is 1 GB of duplicated textures gone (4.84 GB to 3.83 GB of GPU memory). There is a test with eight threads asking for one image.

[untoldengine]

Small design question rather than a bug: since a failed build is now memoized per pack load (per the comment on UntoldBuildCache below), a single transient failure on the first placement of a .untold (e.g. a disk hiccup under the 8-way concurrent load) will now fall through to the fallback mesh for every other placement of that file too, whereas before each placement loaded independently and only that one entity would show the fallback. Sounds like this is intentional (no retries, fail fast) — just flagging in case a one-shot retry inside UntoldBuildCache.build before memoizing failure is worth it, given it only affects the rare transient-I/O case.

[miogds]

It was deliberate, so that a broken file placed hundreds of times is read once, but you're right that it also turned one passing read error into a fallback mesh for every placement, where each placement used to load on its own. In 02f3542 the caller that builds a file tries a second time before the failure is remembered: a passing error costs one more read, and a file that is really broken is read twice and no more. The callers waiting for it get the result of the second try.

[untoldengine]

This claim/waiters/finish dance under a raw NSLock works, but it's a fair amount of machinery for single-flight semantics — AssetDiskCache elsewhere in the codebase gets the same guarantee from a plain actor with a lot less code and no risk of a future edit accidentally resuming a continuation while holding the lock. Might be worth converting this to an actor if you're touching this file again soon. Also, waitOrResult's UntoldBuild?? return type took me a sec to parse (outer optional = "not ready yet", inner optional = "build succeeded but returned nil") — a small enum there instead of nested optionals would make future edits safer. Not blocking, just a maintainability nit.

[miogds]

I took the readability half of this: the nested optional and the bare NSLock are gone. The cache keeps one table of named states (building(waiters:) or finished) inside an OSAllocatedUnfairLock, so the table can only be read or changed while the lock is held, and the class is Sendable without @unchecked (3482d42). There is also a test that two files build at the same time.

I'd rather not make it an actor, though:

• An actor does not give single-flight by itself. Actors are reentrant at every await, so the table of what is in flight stays: RemoteAssetDownloader keeps its inFlight: [URL: Task] for that reason. AssetDiskCache is simpler because it does its work inside the actor, one call at a time.
• That is the part that does not fit here. A build is tens of milliseconds to seconds of parsing and buffer allocation, and the pack loader runs eight at once. Inside the actor they would run one after another. To keep them concurrent, each build would have to become an unstructured task with a @Sendable escaping closure, where today the first caller simply builds on the thread it is already on.
• We measured actors against locks for the engine this week. Crossing into an actor costs about 14 times an unfair lock (47 ns against 3.3 ns, and 2.3 µs against 115 ns with eight callers), and a synchronous thread can only reach one through a task, which waits milliseconds when asset jobs fill the cooperative pool. The rule we took from it: actors stay at the async edges where they are today, nothing the frame reads sits behind one, and the default lock is OSAllocatedUnfairLock, held for a few instructions. This cache is on an async edge, so an actor would be allowed; with the state inside the lock I don't think it would buy anything.

[untoldengine]

Just noting for future-proofing: usesSharedBuild only fires because PackLoadDispatcher always passes assetName: nil and streamingPolicy: .immediate today. If a future pack feature ever needs a non-default assetName or streaming policy while still wanting shared builds, this condition would silently drop back to the unshared path with no compiler signal. Might be worth a comment here (or an explicit canShareBuild flag on the caller side) so it's clear this gate is coupled to the dispatcher's current call pattern.

[miogds]

Good point. The test now says what it depends on (3482d42): a shared build is the whole file built at once, so a load of one named node or a streamed load is not shared even when the caller passes the cache, and that line is the place to change when a pack feature needs either. The doc comment of loadEntityMeshAsync says the same. I kept it a comment rather than a flag on the dispatcher, because the condition is about what the cache can hold, not about what the caller wants.

[miogds]

Thanks for the careful read: all five were worth it. Each has an answer inline, and four commits came out of them:

• f114811: the memory budget counts the buffers and textures that placements share once.
• af6b706: the image cache is single-flight.
• 02f3542: a failed build is tried once more before it is remembered.
• 3482d42: the build cache keeps named states inside an unfair lock, and the test that decides whether a load is shared says what it depends on.

The description is updated to match.