feat(substrate): provision the snapshot store on CAPA through the chart — Crossplane S3 bucket + IRSA role, the snapshot location derived, the role annotation on atelet and ate-api-server

[teemow]

Refs #411 (epic giantswarm/giantswarm#37705, wave 1e; the live proof is the first CAPA installation's cut-over).

What

kagent.harness.snapshotStore.crossplane provisions Agent Substrate's snapshot store on CAPA the way postgres.backup.crossplane provisions the kagent-pg backup bucket:

• Connectivity chart (templates/substrate/crossplane-aws.yaml): the Crossplane Bucket (lifecycle expiration aws.lifecycleDays, 30 days by default — an actor whose snapshot expired boots cold from the golden image again, nothing else is lost; public-access block; TLS-only policy; no Delete management policy, helm.sh/resource-policy: keep) and the IAM Role (aws.roleName, the bucket name by default) trusted through the installation's IRSA OIDC provider for system:serviceaccount:ate-system:atelet and …:ate-api-server, with a list/get/put/delete inline policy on the bucket.
• Helpers reused, not copied: the postgres Crossplane helpers are generalised to take the store block (agent-platform.crossplane.{managementPolicies,managementPoliciesNoDelete,awsPartition,tags}) and the bucket quartet, the trust statement and the S3 policy become shared partials (templates/_crossplane-aws.tpl); the postgres template renders on them byte-identically (all 22 connectivity ci/*.yaml renders diffed before/after: identical).
• Meta chart: derives kagent.harness.snapshotLocation = s3://<aws.bucketName>/<prefix> (prefix kagent by default; no prefix → s3://<bucket>) for the kagent release — an explicit value that disagrees fails the render naming both keys; forwards the role's ARN as eks.amazonaws.com/role-arn on the substrate release's atelet.serviceAccount.annotations and ateApiServer.serviceAccount.annotations (an installation's own annotations stay next to it, a differing explicit role fails the render); holds the block back from the kagent chart — components.kagent.omitKeys now takes dotted paths (agent-platform.omitPath, top-level entries unchanged).
• Guards: aws is the only provider; providerConfigRef, region, aws.{bucketName,accountId,oidcProvider} required when on; accountId must be the quoted 12-digit id (a bare number would print as a float in the ARN).
• Substrate floor: components.substrate / substrate-crds → >=0.0.27-gs.8 <0.0.28-0 (helm: ServiceAccount annotations for atelet and ate-api-server, extraEnv for ate-api-server substrate#27, released v0.0.27-gs.8: atelet.serviceAccount.annotations, ateApiServer.serviceAccount.annotations, ateApiServer.extraEnv); examples/customer-bom.yaml and tests/verify-components.py follow.

Verification

• make verify-substrate-store (new; in .circleci/custom.yml): off by default nothing renders and nothing is derived; on, the bucket quartet + role with both trust statements, the China partition, observeOnly, roleName; the meta chart's derived location, the held-back block, the annotation on both ServiceAccounts next to an operator's own; every guard.
• make verify-postgres verify-postgres-kagent-v2 verify-kagent-harness verify-components verify-meta verify-auto verify-kyverno: green.
• Off shape byte-identical: every helm/agent-platform-connectivity/ci/*.yaml render is unchanged. The meta chart's ci/*.yaml renders differ only by the new default keys reaching the connectivity release's values (its kagent block is open, so a live connectivity chart that predates this release accepts them — the fleet's transition window).
• New CI values file ci/test-substrate-store-aws-values.yaml (the on shape) for the chart's ATS.

Lab (agentlab, platform.chartPath on this checkout)

agentlab (cluster agentlab, meta 4.8.0, connectivity 4.8.1 live) with platform.chartPath on this checkout, the block off (the lab's store is the bundled RustFS, s3://ate-snapshots/kagent):

• agentlab platform: "Installing the local chart at …/helm/agent-platform" — exit 0 in 1m04s; the meta release went to revision 4 (agent-platform-1.1.35, the checkout's placeholder version); 16/16 HelmReleases Ready.
• agentlab platform-test: 9/9 PASS, exit 0.
• The kagent HelmRelease kept harness.snapshotLocation: s3://ate-snapshots/kagent and carries no snapshotStore key (held back); the substrate HelmRelease's atelet / ateApiServer values and the two ServiceAccounts are unchanged (no role annotation derived with the block off).
• The connectivity HelmRelease was upgraded (revision 4) on the live 4.8.1 chart with the new default kagent.harness.snapshotStore block in its values — Ready: True, Helm upgrade succeeded: a connectivity chart that predates this release accepts the new keys, which is the fleet's transition window between the meta chart's roll-out and the connectivity OCIRepository re-resolving.
• Restored: platform.chartPath removed, agentlab platform on 4.8.0 (revision 5, 13s), platform-test exit 0, 16/16 Ready.

The connectivity chart's own changes are not installed by chartPath (a component release from its OCIRepository range); they are verified by render (make verify-substrate-store, the 22 byte-identical ci/*.yaml renders) and by the chart's ATS on this PR.

Not lab-verified

Crossplane and IRSA: the lab has no upbound provider and no AWS account. The Crossplane objects are asserted by render (verify-substrate-store); the live proof is the first CAPA installation's cut-over on the kagent API v2 line (the fleet template renders this block).

Checklist

• Update changelog in CHANGELOG.md.
• Make sure values.yaml and values.schema.json are valid (the kagent block is open in both schemas; helm-docs regenerated).

[circleci-architect]

Published Helm chart agent-platform

4.8.2-dev.feat-substr--crossplane.2026-09-12.22-49-14.hebd593a

Chart	agent-platform
Version	4.8.2-dev.feat-substr--crossplane.2026-09-12.22-49-14.hebd593a
OCI reference	oci://gsoci.azurecr.io/charts/giantswarm/agent-platform:4.8.2-dev.feat-substr--crossplane.2026-09-12.22-49-14.hebd593a
Digest	sha256:a12e51cb3699a0ec943e06efaffc8ca9c4487cb4e27df8c8237082bae646213d
Registry	public — gsoci.azurecr.io
Git catalog	giantswarm-test-catalog (index)

Pull this chart

helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform --version 4.8.2-dev.feat-substr--crossplane.2026-09-12.22-49-14.hebd593a

_{Posted by architect-orb · build 6745 · commit ebd593a · updated in place on every push}

[circleci-architect]

Published Helm chart agent-platform-connectivity

4.8.2-dev.feat-substr--crossplane.2026-09-12.22-49-14.hebd593a

Chart	agent-platform-connectivity
Version	4.8.2-dev.feat-substr--crossplane.2026-09-12.22-49-14.hebd593a
OCI reference	oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity:4.8.2-dev.feat-substr--crossplane.2026-09-12.22-49-14.hebd593a
Digest	sha256:d859d14bb66990d44f6b19344979a1cb63063927abce6da37538d7ec66972c1b
Registry	public — gsoci.azurecr.io
Git catalog	giantswarm-test-catalog (index)

Pull this chart

helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity --version 4.8.2-dev.feat-substr--crossplane.2026-09-12.22-49-14.hebd593a

_{Posted by architect-orb · build 6744 · commit ebd593a · updated in place on every push}