Skip to content

cluster-manager: ship the PriorityClass the pool chart's prewarm placeholder needs — a pool release cannot create cluster-scoped objects as the tenant account, so prewarm: true fails the pool install #539

Description

@teemow

Problem

The gpu-node-pool chart's pool.prewarm (0.4.0) renders a cluster-scoped PriorityClass (value -1000) for its placeholder Job. Pool releases are
delivered by the organisation's tenant ServiceAccount, whose rights are namespaced (Flux multi-tenancy): the release fails InstallFailed on
the PriorityClass and the pool never comes up (create_node_pool {prewarm: true} → release failed 6 s later, no NodeClaim). No negative-priority
class exists on an installation for the Job to reuse.

Root cause

A pool release cannot own a cluster-scoped object, and nothing on the installation ships one the placeholder could name.

Solution

Shipped in #540 → 4.32.0: the connectivity chart renders PriorityClass agent-platform-prewarm-placeholder — value: -1000,
preemptionPolicy: Never, globalDefault: false, a description naming its use — from clusterManager.prewarmPriorityClass {enabled: true, name, value},
mirrored in the meta chart. The class renders with the cluster-manager component, which composes the pool releases and is on in exactly one release
per installation (the platform's): on an installation model serving runs in the serving slice cluster-manager creates beside the pool, so a class gated
on the model-serving switch would arrive after the pool release's Job and leave with the slice. The gpu-node-pool chart names the class through
pool.prewarm.priorityClassName (default: this name) and renders none (giantswarm/gpu-node-pool#15).

Acceptance criteria

Activity

  1. teemow commented on Sep 18, 2026

    @teemow
    MemberAuthor

    Shipped: #540 merged as 66afa47 → v4.32.0 (charts agent-platform:4.32.0@sha256:3c47161b…, agent-platform-connectivity:4.32.0@sha256:bfc4458e…, pullable 152 s after the rerun). The tag pipeline's push-connectivity-chart-release first failed on a GitHub API 502 during the repository-visibility lookup (transient); the original workflow was rerun from the failed job and the chart landed.

    The class lives under clusterManager.prewarmPriorityClass, not modelServing as proposed: on an installation the platform's release runs without the model-serving switch — model serving is the serving slice cluster-manager creates beside the pool — so a class gated on the switch would arrive after the pool release's Job and leave with the slice's teardown. cluster-manager composes the pool releases and is on in exactly one release per installation; the class goes with it (one owner).

    Live on an installation, 2026-09-18 (UTC): platform HelmRelease 4.32.0+3c47161b8afe deployed 01:32:15Z, connectivity 4.32.0+bfc4458ea603 Ready 01:35:20Z; kubectl get priorityclass agent-platform-prewarm-placeholder → value: -1000, preemptionPolicy: Never, globalDefault: false, created 01:35:32Z, meta.helm.sh/release-name: agent-platform-connectivity, app.kubernetes.io/component: cluster-manager, helm.sh/chart: agent-platform-connectivity-4.32.0_bfc4458ea603.

    Open: a pool created with prewarm: true on gpu-node-pool at the release of giantswarm/gpu-node-pool#15 installing and its placeholder pod being admitted — the benchmark run that follows.

  2. changed the title [-]model-serving: ship the PriorityClass the pool chart's prewarm placeholder needs — a pool release cannot create cluster-scoped objects as the tenant account, so prewarm: true fails the pool install[/-] [+]cluster-manager: ship the PriorityClass the pool chart's prewarm placeholder needs — a pool release cannot create cluster-scoped objects as the tenant account, so prewarm: true fails the pool install[/+] on Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions