Add node-local OCI image layer cache: overlay-composed actor rootfs from shared unpacked layers (#463 Phase 1) by dberkov · Pull Request #467 · agent-substrate/substrate

Dmitry Berkovich (dberkov) · 2026-07-20T05:04:47Z

Summary

Phase 1 of #463: replaces atelet's in-memory flattened-tarball pull cache (memorypullcache) with a node-local, on-disk, content-addressed pool of unpacked image layers (internal/imagecache), shared by every actor on the node. An actor's rootfs becomes an overlayfs mount — cached layers as read-only lowerdirs, a bundle-local upper for the actor's private writes — instead of a full image re-untar on every run/resume.

Fixes #437, fixes #166, fixes #228.

What changes

internal/imagecache (new): layer pool (layers/sha256/<diffid>/fs), digest-keyed image records, tag→digest resolution via one remote.Head (tag refs become cacheable), parallel streaming layer download+unpack (bounded, singleflight-collapsed per layer and per image), atomic tmpdir+rename with startup recovery, layout version marker. See internal/imagecache/README.md for the full design.
atelet: prepareOCIDirectory calls EnsureImage and writes a per-bundle rootfs-overlay.json (+ empty rootfs/, upper/, work/) instead of fetch+untar. New --image-cache-dir flag (default /var/lib/ateom-gvisor/image-cache, on the shared hostPath). memorypullcache is deleted.
ateom-gvisor / ateom-microvm: compose the bundle rootfs (SetupBundleRootfs) right before runsc create/restore / virtio-fs lower staging, and detach the mounts at teardown (UnmountAllUnder). Bundles without a spec are left untouched (compatibility with pre-imagecache bundles).
tools/validate-image-cache (new): batch-validates that arbitrary registry images can be pulled, parsed, and unpacked by the store half; used to sweep 400+ real GB-scale eval images during development.
CI: run-tests re-runs the imagecache package under sudo so the root-gated tests (mknod whiteouts, trusted xattrs, overlay mount round trip) execute instead of skipping.

Deviations from the #463 design — please review deliberately

Mounting lives in ateom, not atelet. Design: on-disk, layer-deduplicated OCI image cache for atelet #463 sketched atelet mounting overlays, but atelet deliberately runs with all capabilities dropped ("atelet does no mounts" — manifests/ate-install/atelet.yaml). So the module is split along that existing boundary: atelet pulls/unpacks (pure file I/O) and records whiteouts in per-layer metadata; the privileged ateoms materialize whiteouts (mknod + trusted.overlay.opaque) once per layer and mount, in their own mount namespace — which is where runsc's gofer / virtiofsd resolve paths, so no mount-propagation configuration is needed anywhere.
In-repo untar instead of containerd/archive.Apply. containerd's applier assumes capabilities atelet doesn't have (e.g. writing children into image-defined read-only dirs relies on CAP_DAC_OVERRIDE). The existing hardened untar already solves that (dir-mode juggling for plain root, os.Root confinement, later-entry-wins); it's extended with whiteout capture and missing-parent-dir creation rather than replaced.

Behavior preserved

Pristine rootfs per run: the actor's upper is wiped by resetActorDirs between runs, same contract as the old full re-untar — just at mount cost (ms) instead of extraction cost (tens of seconds).
Micro-VM path nearly untouched: the bundle rootfs it bind-mounts into virtiofsd is now an overlay mountpoint; the guest keeps building its own tmpfs upper. The find-paths requirement (byte-identical lower across reconstructions) holds — cached layers are more deterministic than per-node untars.

Validation

kind e2e (gVisor): counter demo run → increment → suspend → resume; runsc create/start/restore all on overlay rootfs; oci_unpack 2.9–4.8 ms on resume (was ~15–20 s per Avoid re-untarring actor image rootfs on every restore #166); durable state preserved across suspend/resume.
GKE (gVisor + micro-VM): counter and counter-microvm demos, external GCS checkpoints, repeated suspend/resume cycles; cache hits and ms-scale oci_unpack on both runtimes; cache survives atelet restarts.
Image corpus: 411 real production eval images (~1–1.5 GB compressed, up to 19 layers) pulled/parsed/unpacked with 0 failures via tools/validate-image-cache. Two real-image bugs found on GKE during development are fixed with regression tests: layer tars omitting parent-dir entries, and images listing the same layer twice (overlayfs rejects duplicate lowerdirs with ELOOP; compose dedups to the topmost occurrence).
Unit tests: unpack security suite (path traversal, symlink/hardlink escape, whiteout capture), spec round-trips, mountinfo parsing, and root-gated tests for the privileged half — including a full mount → write-isolation → unmount round trip asserting actor writes land in the bundle upper, never in the shared pool.

Known gaps / notes for reviewers

No GC yet (Phase 2 of Design: on-disk, layer-deduplicated OCI image cache for atelet #463): layers are never evicted; disk grows with the set of distinct layers ever pulled. Documented in internal/imagecache/README.md with operator guidance.
Upgrade ordering: a new atelet with old ateom worker pods yields an empty (unmounted) rootfs — deploy ateoms with/before atelet. Old bundles keep working under new ateoms.
The untar tracing span no longer exists (untar is gone); prepareOCIDirectory and RPC spans remain. An EnsureImage span with hit/miss attributes is natural Phase 2 material alongside cache metrics.

🤖 Generated with Claude Code

Benjamin Elder (BenTheElder)

Quick AI review ... (continuing to refine that experiment on coworker PRs for now, intend to add a skill)

Findings inline: two 🟡 (overlayfs mount-option page-size cap → images beyond ~34 layers can't mount; phantom parent dirs shadowing lower-layer dir metadata in the merged view) and two 🟢 (failure-path unmount asymmetry; record temp files not swept).

Checked and found no issues: unpack confinement (os.Root, tar-name validation, hardlink/symlink replacement) matches the old hardened untar; whiteout capture→materialize split is correct and FinalizeLayer is safely idempotent under concurrent ateoms; lowerdir dedup (topmost wins) is the right fix for the duplicate-layer ELOOP; the tag→digest HEAD resolution only affects non-pinned refs (actor images are digest-pinned by CRD validation); microvm covers both run and restore via buildActorContainers; FIFO/device tar entries still hard-fail but did in the old untar too (pre-existing, not a regression); GOOS=linux build and the unprivileged imagecache tests pass locally.

Drafted with the assistance of an automated review agent; reviewed by a human before submitting.

Benjamin Elder (BenTheElder) · 2026-07-20T20:15:59Z

+	if strings.ContainsAny(upper, ":,") || strings.ContainsAny(work, ":,") {
+		return "", fmt.Errorf("bundle path %q contains overlay option separators", upper)
+	}
+	return "lowerdir=" + strings.Join(lowers, ":") + ",upperdir=" + upper + ",workdir=" + work, nil


🟡 🤖 No guard on the assembled option-string length. mount(2) copies options via a single page, so the whole string is capped at ~4096 bytes. Layer paths here are ~114 bytes each (/var/lib/ateom-gvisor/image-cache/layers/sha256/<64-hex>/fs:), which caps images at roughly 34 layers — Docker permits 127, and heavily-layered CI/base-chain images in the 30–60 range exist in the wild. The validation corpus topped out at 19 layers, so this wasn't hit; when it is, the failure is an opaque EINVAL from the mount.

This is exactly why containerd's overlay snapshotter uses short integer snapshot IDs. Options: a preflight length check with a clear error (minimum), short per-layer symlinks/IDs in the pool, or the new mount API (fsconfig + lowerdir+ appends, kernel ≥ 6.5) which avoids the single-page limit.

Fixed in 7702652 by switching to the new mount API: fsopen + one fsconfig lowerdir+ append per layer removes the aggregate option-string limit structurally (no symlink indirection needed), and failed mounts now surface the kernel's fs-context error log instead of a bare EINVAL. Added a 64-layer root-gated regression test that asserts the joined paths exceed a page before mounting. This sets a kernel floor of 6.5 (lowerdir+) — documented in the code and README; all current GKE channels ship ≥6.6, and we verified empirically that a 6.1 kernel (debian-12) fails fast with a clear kernel log line rather than corrupting anything.

Benjamin Elder (BenTheElder) · 2026-07-20T20:15:59Z

+		// lower layers' directories at compose time.
+		if parent := filepath.Dir(name); parent != "." {
+			if err := root.MkdirAll(parent, 0o755); err != nil {
+				return nil, fmt.Errorf("while creating parent directories for %q: %w", name, err)


🟡 🤖 Phantom parent dirs shadow lower-layer directory metadata in the merged view. In overlayfs, the top-most layer containing a directory supplies its attributes. A parent auto-created here (root:root 0755) in an upper layer hides the real metadata a lower layer declared — e.g. a layer shipping root/.bashrc without a root/ entry turns /root from 0700 into 0755; a file added under /tmp without a tmp/ entry drops the 1777 sticky bit. The old flattened extract didn't have this (parents declared in any layer kept their modes); the PR notes the omitted-parent case was hit on real GKE images, so this will occur. mknodWhiteout's MkdirAll in bundle_linux.go has the same property.

Not easily fixable without lower-layer context at unpack time (containerd sidesteps it by applying through the mounted parent chain, so copy-up preserves metadata). At minimum worth documenting in the README as a known deviation; a fuller fix could consult the already-cached lower layers of the same image when creating phantom parents.

Implemented the fuller fix in 94fb9ef rather than document-only. Baking lower-derived attrs into the pool is unsound (layer trees are shared across images with different chains), so the repair is per-actor at compose time: unpack records implicitly-created dirs in the layer metadata (including parents that only exist for whiteout materialization — the mknodWhiteout MkdirAll case you noted), and SetupBundleRootfs resolves each shadowed dir's true mode+ownership from the top-most non-implicit layer in that image's chain, applying it through the mount so the copy-up lands in the bundle's private upper. The shared pool is never modified (asserted in the root-gated test). Pre-existing cached layers have no recorded metadata and are simply left as-is. Residual gaps (mtimes, xattrs, dirs implicit in every layer) are documented in the README.

Benjamin Elder (BenTheElder) · 2026-07-20T20:15:59Z

+	// (deleting a bundle out from under a live mount in this namespace would
+	// leave the mount orphaned until the pod restarts). Best-effort, same as
+	// the container cleanup above.
+	if err := imagecache.UnmountAllUnder(ateompath.OCIBundleDir(req.GetActorUid())); err != nil {


🟢 🤖 Teardown asymmetry: this detaches bundle overlays after checkpoint, but the Run-failure defer (line ~197) and Restore-failure defer (~390) only clean up the network — overlays mounted by a partially-successful SetupBundleRootfs linger in this namespace until atelet's later bundle wipe lazily detaches them. Adding UnmountAllUnder to those failure defers would make cleanup symmetric.

Fixed in 91a2f0d (gvisor) and d5787a9 (microvm — buildActorContainers mounts the same way, so its Run and Restore failure defers had the identical gap). In gvisor the detach runs before cleanupActorNetworkOrExit, which exits the process on failure and would otherwise skip it.

Benjamin Elder (BenTheElder) · 2026-07-20T20:15:59Z

+// sweepTempDirs removes unpack temp dirs orphaned by a crash. A layer dir
+// without the temp prefix is always complete (it was moved into place with a
+// single rename), so this is the only recovery the pool needs.
+func (s *Store) sweepTempDirs() error {


🟢 🤖 This sweeps only orphaned layer temp dirs; writeRecord's temp files (.<digest>.json.tmp-* under manifests/sha256/) orphaned by a crash are never cleaned. Tiny files, but the sweep exists for exactly this class — worth including the manifests dir.

Fixed in f7970cf — the startup sweep now also removes writeRecord's "..json.tmp-*" orphans (finished records never start with a dot, so the check is exact), with the recovery test extended to cover both the orphan removal and a completed record surviving the sweep.

Startup recovery swept layer unpack temp dirs but not writeRecord's ".<digest>.json.tmp-*" files left by a crash between create and rename. Addresses PR agent-substrate#467 review feedback.

Layer tars omit parents that exist in lower layers; unpack fabricates them (root:root 0755) and overlayfs takes merged dir attrs from the top-most layer containing the dir — so a fabricated parent shadowed real lower-layer metadata (/tmp's 1777, /root's 0700). Record implicit dirs in the layer metadata at unpack, and at compose repair the merged view from the top-most non-implicit provider in the image's chain; the chown/chmod copy-ups land in the bundle's private upper, never in the shared pool. Residual gaps (mtimes, xattrs, implicit-everywhere dirs) documented in the README. Addresses PR agent-substrate#467 review feedback (phantom parent dirs shadowing lower- layer directory metadata).

Startup recovery swept layer unpack temp dirs but not writeRecord's ".<digest>.json.tmp-*" files left by a crash between create and rename. Addresses PR agent-substrate#467 review feedback.

Layer tars omit parents that exist in lower layers; unpack fabricates them (root:root 0755) and overlayfs takes merged dir attrs from the top-most layer containing the dir — so a fabricated parent shadowed real lower-layer metadata (/tmp's 1777, /root's 0700). Record implicit dirs in the layer metadata at unpack, and at compose repair the merged view from the top-most non-implicit provider in the image's chain; the chown/chmod copy-ups land in the bundle's private upper, never in the shared pool. Residual gaps (mtimes, xattrs, implicit-everywhere dirs) documented in the README. Addresses PR agent-substrate#467 review feedback (phantom parent dirs shadowing lower- layer directory metadata).

…llcache Content-addressed pool of unpacked image layers, shared by every actor on the node; actor rootfs becomes an overlayfs mount (cached layers as read- only lowers, bundle-local upper) instead of a full re-untar per run. atelet (no capabilities) pulls and unpacks; the privileged ateoms finalize whiteouts and mount. Tag refs resolve via one HEAD and become cacheable; pull memory is O(stream buffers); the cache survives restarts. Phase 1 of agent-substrate#463. Fixes agent-substrate#437, agent-substrate#166, agent-substrate#228. Validated: kind + GKE counter demos (gvisor and microvm), suspend/resume (oci_unpack ~3ms vs ~15-20s), 411 SWE-bench-scale images pulled and unpacked with 0 failures, root-gated unit tests for the privileged paths. Known gaps: no GC yet (Phase 2, see internal/imagecache/README.md); upgrade ordering — deploy new ateoms before/with the new atelet.

The fixed 30-minute idle guard made nothing evictable while a fast corpus filled a small disk (32-VM hicard sweep: local SSDs filled in ~12 minutes, then every remaining image failed ENOSPC). Expose it as --evict-idle.

Unmount runs before cleanupActorNetworkOrExit, which exits the process on failure and would otherwise skip the overlay detach.

…ayer cap mount(2) copies its option string through a single page, which caps digest-derived lowerdir chains (~114 bytes per layer path) at roughly 34 layers and fails with a bare EINVAL beyond that. Appending lowerdirs one fsconfig(2) call at a time removes the aggregate limit structurally, and failed mounts now carry the kernel's fs-context error log instead of an opaque errno. Adds a 64-layer regression test that asserts the joined paths exceed one page before mounting. Minimum supported kernel: Linux 6.5 (overlayfs "lowerdir+"). All current GKE channels meet it: Stable runs COS 121 LTS (kernel 6.6), Regular and Rapid run COS 125/129 (kernel 6.12).

Startup recovery swept layer unpack temp dirs but not writeRecord's ".<digest>.json.tmp-*" files left by a crash between create and rename. Addresses PR agent-substrate#467 review feedback.

Layer tars omit parents that exist in lower layers; unpack fabricates them (root:root 0755) and overlayfs takes merged dir attrs from the top-most layer containing the dir — so a fabricated parent shadowed real lower-layer metadata (/tmp's 1777, /root's 0700). Record implicit dirs in the layer metadata at unpack, and at compose repair the merged view from the top-most non-implicit provider in the image's chain; the chown/chmod copy-ups land in the bundle's private upper, never in the shared pool. Residual gaps (mtimes, xattrs, implicit-everywhere dirs) documented in the README. Addresses PR agent-substrate#467 review feedback (phantom parent dirs shadowing lower- layer directory metadata).

Benjamin Elder (BenTheElder) · 2026-07-22T22:35:41Z

      with:
        go-version-file: 'go.mod'
    - run: go test -v ./...
+    # The imagecache consumer half (overlay mounts, whiteout mknod, trusted.*


I think we should generalize rootful "integration" tests as a follow-up TODO.

Agreed — filed #501 to generalize rootful test execution (discovery via build tag/convention + a single sudo step, plus contributor docs), so this per-package step becomes the first consumer rather than a pattern.

Benjamin Elder (BenTheElder) · 2026-07-22T22:37:38Z

+// mount(2)'s single-page option-string cap, which digest-derived layer paths
+// (~114 bytes each) would hit at roughly 34 layers.
+//
+// Minimum supported kernel: Linux 6.5, where overlayfs gained the


🟢 🤖 The COS citation covers GKE's default images, but not GKE Ubuntu node images or developer hosts running kind (Ubuntu 22.04 GA is kernel 5.15). On those, every mount now fails at lowerdir+. If they're in scope, consider a fallback to legacy mount(2) when the kernel lacks lowerdir+, or at least a startup probe that fails atelet/ateom with a clear "kernel too old" error instead of per-actor mount failures.

I think we can follow-up on this one as well (green = non-blocking)

Good catch on the non-COS environments — filed #500 for the follow-up: probe lowerdir+ once at ateom startup, fall back to legacy mount(2) assembly when absent (graceful: pre-6.5 kernels can't mount >~34-layer chains by any mechanism, so the fallback only fails where the kernel itself is the limit), and emit one clear kernel-too-old log line instead of per-actor EINVALs.

Startup recovery swept layer unpack temp dirs but not writeRecord's ".<digest>.json.tmp-*" files left by a crash between create and rename. Addresses PR #467 review feedback.

Layer tars omit parents that exist in lower layers; unpack fabricates them (root:root 0755) and overlayfs takes merged dir attrs from the top-most layer containing the dir — so a fabricated parent shadowed real lower-layer metadata (/tmp's 1777, /root's 0700). Record implicit dirs in the layer metadata at unpack, and at compose repair the merged view from the top-most non-implicit provider in the image's chain; the chown/chmod copy-ups land in the bundle's private upper, never in the shared pool. Residual gaps (mtimes, xattrs, implicit-everywhere dirs) documented in the README. Addresses PR #467 review feedback (phantom parent dirs shadowing lower- layer directory metadata).

Dmitry Berkovich (dberkov) mentioned this pull request Jul 20, 2026

Design: on-disk, layer-deduplicated OCI image cache for atelet #463

Open

13 tasks

Dmitry Berkovich (dberkov) requested review from Benjamin Elder (BenTheElder) and Julian Gutierrez Oschmann (juli4n) July 20, 2026 05:23

Dmitry Berkovich (dberkov) force-pushed the image-cache-phase1 branch from 9450dcf to 8933927 Compare July 20, 2026 19:25

Benjamin Elder (BenTheElder) reviewed Jul 20, 2026

View reviewed changes

Dmitry Berkovich (dberkov) force-pushed the image-cache-phase1 branch from c84cef5 to 1d39b78 Compare July 21, 2026 03:39

Dmitry Berkovich (dberkov) added 7 commits July 20, 2026 20:43

validate-image-cache: make eviction idle window configurable

92048b1

The fixed 30-minute idle guard made nothing evictable while a fast corpus filled a small disk (32-VM hicard sweep: local SSDs filled in ~12 minutes, then every remaining image failed ENOSPC). Expose it as --evict-idle.

gvisor: detach bundle rootfs overlays on Run/Restore failure paths

91a2f0d

Unmount runs before cleanupActorNetworkOrExit, which exits the process on failure and would otherwise skip the overlay detach.

microvm: detach bundle rootfs overlays on Run/Restore failure paths

d5787a9

imagecache: sweep orphaned manifest record temp files at startup

f7970cf

Startup recovery swept layer unpack temp dirs but not writeRecord's ".<digest>.json.tmp-*" files left by a crash between create and rename. Addresses PR agent-substrate#467 review feedback.

Dmitry Berkovich (dberkov) force-pushed the image-cache-phase1 branch from 1d39b78 to 94fb9ef Compare July 21, 2026 03:44

imagecache: generalize duplicate-layer comment wording

1c7a77b

Benjamin Elder (BenTheElder) reviewed Jul 22, 2026

View reviewed changes

This was referenced Jul 23, 2026

imagecache: probe for overlayfs lowerdir+ support and fall back to legacy mount(2) on pre-6.5 kernels #500

Open

CI: generalize rootful (integration) test execution beyond the imagecache one-off #501

Closed

atelet: add successful RPC boundary tests #469

Open

Benjamin Elder (BenTheElder) approved these changes Jul 23, 2026

View reviewed changes

Benjamin Elder (BenTheElder) merged commit 961883a into agent-substrate:main Jul 23, 2026
11 checks passed

feifeigood (feifeigood) mentioned this pull request Jul 23, 2026

deps: bump substrate to 961883a8 (imagecache Phase 1, root-cures atelet OOM) feifeigood/ax#11

Closed

Uh oh!

Conversation

Dmitry Berkovich (dberkov) commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changes

Deviations from the #463 design — please review deliberately

Behavior preserved

Validation

Known gaps / notes for reviewers

Uh oh!

Benjamin Elder (BenTheElder) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dmitry Berkovich (dberkov) Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dmitry Berkovich (dberkov) Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dmitry Berkovich (dberkov) Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dmitry Berkovich (dberkov) Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Dmitry Berkovich (dberkov) commented Jul 20, 2026 •

edited

Loading

Dmitry Berkovich (dberkov) Jul 21, 2026 •

edited

Loading

Dmitry Berkovich (dberkov) Jul 21, 2026 •

edited

Loading

Dmitry Berkovich (dberkov) Jul 21, 2026 •

edited

Loading

Dmitry Berkovich (dberkov) Jul 21, 2026 •

edited

Loading