Skip to content

Target kubeconfig: one chart release per workload cluster, slices as toggles #328

Closed

Description

@teemow

Problem

The serving slice (#326) and the runtime slice (#317) are values profiles of this chart that cluster-manager installs as a HelmRelease of its own — beside the platform's own release on the installation's cluster, or onto a workload cluster that has no platform at all (bumblebee-plans#46 round 3, Timo 2026-09-09: the model serving stack is installed via a HelmRelease in every case where the platform's own release does not provide it; Add GPU node pool on a workload cluster creates the pool release, the gpu-operator release if needed and the <cluster>-agent-platform slice release if needed, all on the installation; round 4 — bumblebee-plans#46 (round 4, 2026-09-09): one release of this chart per target cluster, named <cluster>-agent-platform, with the slices as toggles in its values — created with the first slice, its values updated in place when the second is enabled; never two releases of the chart on one cluster, since both slices need agentgateway's cluster-scoped CRDs). Three things in the chart stand in the way:

  1. The component loop renders every component as an OCIRepository + HelmRelease in the release namespace with no kubeConfig, so a chart installed through a kubeconfig-targeted HelmRelease puts Flux CRs onto the target cluster where nothing reconciles them. The only way out today is the bundled Flux engine on the workload cluster — which Timo ruled out: never install Flux into a cluster that runs one, and never into a workload cluster; the installation's Flux reconciles workload-cluster apps, as the fleet does for every operations app through HelmRelease.spec.kubeConfig.secretRef (workload-clusters-fleet/…/operations/apps/*/helm-release.yaml).
  2. muster, dicebear and the connectivity chart are force-on (no enabled key), so a slice release beside the platform's release would run a second muster — the contract question Runtime slice values example: kagent only, for workload clusters that run agents managed from the installation (a kubeConfig-targeted HelmRelease of this chart) #317 left open.
  3. Cluster-scoped objects (the KServe CRDs, agentgateway's CRDs) can have exactly one owner on a cluster.

Proposed solution

  • Target knob — gitops.target.kubeConfigSecretRef (name to settle): when set, the component loop stamps spec.kubeConfig.secretRef onto every component HelmRelease (the connectivity release included), so the installation's helm-controller installs each component into the target cluster from the OCIRepositories in the release namespace: one set of sources on the installation, dependsOn intact, install.createNamespace creating the target namespaces there, the connectivity chart's lookup/Capabilities guards evaluated against the target — where they belong. components.flux.enabled must be false with the knob (render guard).
  • enabled toggles for muster and dicebear (default true) so the slice profiles turn them off; the connectivity chart stays always-on but renders only what enabled components need (it already drops references at render time). Runtime slice values example: kagent only, for workload clusters that run agents managed from the installation (a kubeConfig-targeted HelmRelease of this chart) #317's open question, decided. components.agentgateway already has enabled (default false) — the slice profiles set it per target: on for a workload cluster, off beside the platform's release on the installation's own cluster, which already owns the controller and its CRDs.
  • Beside the platform's release: a slice release names its own release namespace; CRD components carry helm.sh/resource-policy: keep and belong to whichever release installs them; the render refuses a component whose cluster-scoped objects another release owns and names the release to remove first (the validateEngine pattern; adoption set aside) — one owner per component per cluster. The lookup guards see the cluster the meta chart renders against: for the platform's own release that is its own cluster — the only place a toggle and cluster-manager's release can meet; on the kubeconfig-targeted path the meta chart renders on the installation while its components land on the target, so there detection is cluster-manager's (bumblebee-plans#46 (round 4, 2026-09-09)).
  • Engine guard exercised in CI: fixtures with a foreign helm-controller Deployment (engine on → fail) and with the knob set and the engine on (→ fail).
  • One release per target cluster — the shape the toggles serve: examples/serving-slice.yaml (Serving slice: models Gateway, llmisvc, presets, kserve-runtime-configs #326) and examples/runtime-slice.yaml (Runtime slice values example: kagent only, for workload clusters that run agents managed from the installation (a kubeConfig-targeted HelmRelease of this chart) #317) are composable value sets of one <cluster>-agent-platform release; both set the knob for the workload-cluster case; a release whose slice toggles change in place (a second slice switched on, one switched off) upgrades cleanly. make verify renders each profile with and without the knob, and the two combined.

Acceptance criteria

  • With the knob set, every rendered HelmRelease carries the kubeConfig secretRef; without it the render is byte-identical to today.
  • components.muster.enabled: false and components.dicebear.enabled: false render nothing of them and the connectivity release drops their references; components.agentgateway.enabled follows the slice profile per target; the default render is byte-identical.
  • Guards: engine on + knob fails; a foreign helm-controller + engine on fails; a second owner of the KServe or agentgateway CRDs fails naming the release to remove first.
  • The serving and runtime profiles render combined into one release and each alone; switching a slice on in an existing release is an in-place upgrade (fixture).
  • The epic's cloud proofs install the serving slice through the knob from the installation — no Flux on the workload cluster (giantswarm/giantswarm#37715 on gazelle's operations; GPU proof on the lab-created cloud cluster agentlab#128 on the lab-created cluster).

Proposed in bumblebee-plans#46 (round 3 of 2026-09-09; the PRD's D3/D4/D5/D7). Epic: giantswarm/giantswarm#37639. Related: #326 (serving slice — consumer), #317 (runtime slice — consumer), giantswarm/giantswarm#37637 (cluster-manager composes the release).

Activity

  1. moved this from Inbox 📥 to Backlog 📦 in Roadmapon Sep 9, 2026
  2. changed the title [-]The chart as a slice release: a target kubeconfig knob stamped on every component HelmRelease so the installation's helm-controller installs the components into a workload cluster (no Flux there), enabled toggles for muster and dicebear, one owner per cluster-scoped component, the engine guard exercised in CI — the delivery the serving slice (#326) and the runtime slice (#317) use (bumblebee-plans#46 round 3, D4)[/-] [+]The chart as a slice release: a target kubeconfig knob stamped on every component HelmRelease so the installation's helm-controller installs the components into a workload cluster (no Flux there), enabled toggles for muster and dicebear, one release of the chart per target cluster with the slices as toggles, one owner per cluster-scoped component, the engine guard exercised in CI — the delivery the serving slice (#326) and the runtime slice (#317) use (bumblebee-plans#46 round 3, D4)[/+] on Sep 9, 2026
  3. changed the title [-]The chart as a slice release: a target kubeconfig knob stamped on every component HelmRelease so the installation's helm-controller installs the components into a workload cluster (no Flux there), enabled toggles for muster and dicebear, one release of the chart per target cluster with the slices as toggles, one owner per cluster-scoped component, the engine guard exercised in CI — the delivery the serving slice (#326) and the runtime slice (#317) use (bumblebee-plans#46 round 3, D4)[/-] [+]Target kubeconfig: one chart release per workload cluster, slices as toggles[/+] on Sep 15, 2026
  4. moved this from Backlog 📦 to In Progress ⛏️ in Roadmapon Sep 15, 2026
  5. moved this from In Progress ⛏️ to Validation ☑️ in Roadmapon Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    • Status
      Done ✅

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions