QuentinBisson · Oct 9, 2026 · Oct 9, 2026 · Oct 9, 2026
@@ -75,7 +75,7 @@ Everything on `giantswarm` that is not in the pin (`git log v0.4.0-alpha1..giant
 | The bundled RustFS Deployment stops its old pod before it starts the new one (`RollingUpdate` with `maxSurge: 0` and `maxUnavailable: 1`) | it runs one replica on the ReadWriteOnce claim `rustfs-data`, and the default RollingUpdate starts the new pod while the old one still holds the volume, which ReadWriteOnce attaches to one node: a new pod scheduled on another node stays `ContainerCreating` (`FailedAttachVolume`, the volume already used by the old pod), the old one is never scaled down, and the Substrate release waiting on the rollout stalls until the old pod is deleted by hand (an installation's upgrade, [giantswarm/giantswarm#37742](https://github.com/giantswarm/giantswarm/issues/37742) row 69). Not `Recreate`, which the patch first carried: an existing Deployment keeps the API server's defaulted `rollingUpdate`, which no field manager owns, so helm-controller's server-side apply of `type: Recreate` cannot drop it and the API server refuses the upgrade (`spec.strategy.rollingUpdate: Forbidden: may not be specified when strategy type is 'Recreate'`, a server-side dry-run over an installation's 1.1.0 Deployment; an explicit `rollingUpdate: null` is refused the same way). No surge and one unavailable apply over the defaulted strategy and let a rollout already stuck with two pods finish. The bundled Postgres is a StatefulSet and replaces its pod delete-first. `charts/substrate/tests/rustfs_test.yaml` holds the strategy | `bff94a57` ([#76](https://github.com/giantswarm/substrate/pull/76), squash: `Recreate`) and `744bd4d7` ([#83](https://github.com/giantswarm/substrate/pull/83), no surge instead) | to file: the upstream-shaped patch is branch [`upstream/rustfs-recreate`](https://github.com/giantswarm/substrate/tree/upstream/rustfs-recreate) here (`fbcefb79` the template hunk, `8634c2bf` the rendered `manifests/ate-install/rustfs.yaml`, `0d834268` the comment's per-node wording, `e10b45af` no surge instead of `Recreate`, on the mirror `main` @ `8316538f`, DCO-signed, no fork references; the first message is the pull-request text; the chart test stays fork-only, upstream has no chart tests), for kagent-dev/substrate; a team member opens it with DCO sign-off once #37742 has reviewed it; giantswarm/giantswarm#37742 row 69 |
 | The `rustfs-bucket-init` Job is a Helm hook (`post-install,post-upgrade`, delete policy `before-hook-creation,hook-succeeded`: a completed Job left behind would block a rollback to a release whose Job is a plain resource of the same name; `hack/install-microvm-deps.sh` and `hack/microvm-assets/stage-to-rustfs.sh` wait for the Job only while it exists), recreated on every install and upgrade under the same name, and `images.awsCli` is the gsoci copy `gsoci.azurecr.io/giantswarm/aws-cli:2.17.0` (upstream's digest) | a Job's pod template is immutable: as a plain release resource any change of its image failed the upgrade of every installation with `spec.template: … field is immutable` until the completed Job was deleted by hand, so the chart kept Docker Hub's short name `amazon/aws-cli` and the platform could not run from gsoci only (giantswarm/agent-platform#580). On an upgrade from a release where the Job is a release resource, Helm deletes the old Job (gone from the manifest) and the post-upgrade hook creates the new one. `charts/substrate/tests/rustfs_test.yaml` holds the hook | `f4256587` ([#85](https://github.com/giantswarm/substrate/pull/85)) | to file for kagent-dev/substrate (the hook only; the gsoci default is fork-only), [giantswarm/giantswarm#37742](https://github.com/giantswarm/giantswarm/issues/37742) row 94 |
 | Actors run as their image's `USER` in its `WORKDIR` (atelet resolves `USER` as Docker does, against the image's own `/etc/passwd` and `/etc/group`; no `USER` is root; the pause container follows the same rule; the process starts in `WORKDIR`, default `/`, created in the actor's upper when the image lacks it), and what a non-root actor needs: the bundle's `rootfs` and overlay `upper` are `0755`, a `durableDir` volume is `0777`, and the kubectl manifest's atelet holds `CAP_DAC_OVERRIDE` to empty the directories a non-root actor left in it (the chart's atelet is privileged already) | atelet ran every container as root in `/` whatever its image declared, and on gVisor the container's `/` took the `0700` of the overlay upper, so a process that is not root could exec nothing. kagent's harness images declare `USER 65532` and were run as root regardless; giantswarm/kagent-upstream#125 separates the harness (root, declared as such in the image) from Claude Code and Codex (uid 65532) ([#37742](https://github.com/giantswarm/giantswarm/issues/37742) row 85, giantswarm/giantswarm#38054) | `git cherry-pick -x` of the whole stack, in upstream's order: `f55a4de4` (upstream's `8f7eb07d`, rootfs root), with the fork-only test of a restored bundle `f4a98eb1`; `1f43f481` (`895ec3ce`, #1906); `12670632`, `3da2115b`, `136337e7`, `ab5c083e` (`1e0466df`, `fa8578de`, `5925173f`, `d41a2fc2`, #1918). `afb62e18` (agent-substrate/substrate#1910, the manifest's `CAP_DAC_OVERRIDE`) is in the pin. Adapted to the line: `ocispec.Options` keeps its `ActorUID`/`ContainerName` shape and gains only `User` and `Cwd`, paths stay in `ateompath`, and the fork's `TestPrepareOCIDirectory_RegistryRejectionIsTerminal` passes the bundle path | not ours: [agent-substrate/substrate#1918](https://github.com/agent-substrate/substrate/pull/1918) (author dims, open, five commits, rebased on 2026-10-01) and [#1906](https://github.com/agent-substrate/substrate/pull/1906) (open), which #1918 asks to merge first; #1910 is merged. All fall away at the re-pin onto the first release that carries #1918. **The upgrade onto this line breaks every gVisor snapshot**: `runsc restore` checks each container's `Process.User` and `Process.Cwd` against the checkpoint and the pause image sets `USER 65535`, so delete every gVisor actor before the roll and recreate every gVisor template after it (`docs/upgrade.md`, step 4); micro-VM snapshots resume as they were. The harness images must declare their identity (`USER 0:0`, `WORKDIR /`, kagent-upstream#125) before the roll, or the harness itself starts as 65532 |
-| Image-layer files keep the owner their layer tar gives them (atelet records the tar's non-root owners, with the setuid/setgid mode a chown clears, in the layer metadata `whiteouts.json` from its `version` 2; ateom's `FinalizeLayer` applies them to the pooled layer tree with `lchown` under its own `owners-applied` marker, once per layer node-wide, so the merged rootfs presents them with no copy-up; a layer pooled earlier is backfilled from its tar headers when a pull reuses it) | atelet unpacks with every capability dropped, so every image-layer file belonged to root: an actor run as its image's `USER` could not write what its image gave it. kagent's Go ADK image declares `USER 65532` and its golden boot failed with `mkdir /config: permission denied`, then `open /config/config.json: permission denied` once the image created `/config` for its user (giantswarm/kagent-upstream#125, giantswarm/giantswarm#38054) | [#115](https://github.com/giantswarm/substrate/pull/115) | to file for agent-substrate/substrate#2036, which #1918 needs before it merges |
+| Image-layer files keep the owner their layer tar gives them (atelet records the tar's non-root owners, with the setuid/setgid mode a chown clears, in the layer metadata `whiteouts.json` from its `version` 2; ateom's `FinalizeLayer` applies them to the pooled layer tree with `lchown` under its own `owners-applied` marker, once per layer node-wide, so the merged rootfs presents them with no copy-up; a layer pooled earlier is backfilled from its tar headers when a pull reuses it) | atelet unpacks with every capability dropped, so every image-layer file belonged to root: an actor run as its image's `USER` could not write what its image gave it. kagent's Go ADK image declares `USER 65532` and its golden boot failed with `mkdir /config: permission denied`, then `open /config/config.json: permission denied` once the image created `/config` for its user (giantswarm/kagent-upstream#125, giantswarm/giantswarm#38054) | [#115](https://github.com/giantswarm/substrate/pull/115) | not ours: [agent-substrate/substrate#2392](https://github.com/agent-substrate/substrate/pull/2392) (author aramase, open, `Fixes #2036`, on top of #1918). Falls away at the re-pin onto the first release that carries #2392 |
 | Every gVisor WorkerPool runs the release's own `ateom-gvisor` (atecontroller `--gvisor-worker-image` replaces a gVisor pool's `spec.workerImage` in the worker Deployment; the chart sets it to `ateom-gvisor` composed like the other component images, `ateController.gvisorWorker.followRelease`, on by default; micro-VM pools and the `spec.workerImage` field are unchanged; helm-e2e builds `ateom-gvisor` and points the counter pool at `1.3.0` before the gVisor suites) | atelet and ateom share a node-local protocol with no cross-version guarantee, but the platform set the pool's worker to the floor of its Substrate range while Flux moved atelet to the newest release in it: atelet `1.3.1-rc.4` (the `USER` row above) with worker `1.3.0` left the pause container's `/` at `0700`, and every gVisor actor failed with `failed to load /pause: permission denied` | [#114](https://github.com/giantswarm/substrate/pull/114) | fork-only: upstream installs pin atelet and ateom to one build per node (agent-substrate/substrate#1357); [#37742](https://github.com/giantswarm/giantswarm/issues/37742) row 105 |
 | The egress gateway tells the actor why a credential was refused (`mapCredentialProviderError` in `cmd/atenet/internal/router/egress/credentials.go`, the contract on `FetchSecret` in `pkg/proto/credproviderpb/credprovider.proto`: `NotFound`, `PermissionDenied`, `FailedPrecondition` and `Unauthenticated` deny as 403 with the provider's message, escaped and cut to 512 bytes; `Unavailable`, `DeadlineExceeded` and `ResourceExhausted` as 503; anything else, and an empty or unusable secret, as 502; every body names the provider and nothing of the transport) | kagent's caller credential provider refuses for reasons the actor can act on (no turn running, no grant at the token broker: sign in and retry), and every refusal reached the actor as `egress denied`; git prints a `text/plain` error body as `remote:` lines, so the reason is what a coding agent relays to its person. A provider bug answered 403 like a refusal, which an agent reads as a permission problem: 502 is the gateway's own outcome when the provider it relies on misbehaves. This departs from upstream's deliberate 403 for anything unexpected (chosen so a client does not retry); no answer becomes 404 or 400, which git and `gh` read as a missing repository or a malformed request. The agentgateway dataplane applies the same table (giantswarm/agentgateway-upstream `fork/credential-provider-errors`); giantswarm/giantswarm#38056 | `feat/credential-provider-errors` | to file, with the agentgateway half: the 502 for unexpected answers needs upstream's agreement |
 | The egress gateway's ext-proc takes the chart's `otel` settings like every other Go component (`substrate.otel.env` and `OTEL_RESOURCE_ATTRIBUTES` on the `atenet-egress` `ext-proc` container; a chart unit test) | the container rendered no `OTEL_*` variable, so its SDK exported traces and metrics to the default `localhost:4317` whatever `otel` said: a cluster without a collector logged `OpenTelemetry SDK error … dial tcp [::1]:4317` once a minute, and with one its spans never reached it ([giantswarm/agent-platform#463](https://github.com/giantswarm/agent-platform/issues/463)) | `126a8682` ([#106](https://github.com/giantswarm/substrate/pull/106)) | upstream-ready for [kagent-dev/substrate](https://github.com/kagent-dev/substrate), which ships the chart (its `charts/substrate/templates/atenet-egress.yaml` has the same gap); waits for a free slot there, with kagent-dev/substrate#46 and #48 open; queued in [#37742](https://github.com/giantswarm/giantswarm/issues/37742) |