Skip to content

fix(model-manager): reach the workload clusters' API servers - #689

Merged
teemow merged 1 commit into
mainfrom
model-manager-egress-workload-clusters-687
Sep 25, 2026
Merged

teemow merged 1 commit into
mainfrom
model-manager-egress-workload-clusters-687

Conversation

@teemow

@teemow teemow commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

What does this PR do?

Towards #687 (its live list_backends criterion waits for the next GPU window). A kserve backend that cluster-manager registers targets a workload cluster's apiserver, which model-manager calls as the caller. model-manager's egress policy had no rule to those API servers, so under the cilium flavour list_backends timed out dialing the cluster's API server (dial tcp <ELB>:443: i/o timeout), and so did every tool naming that backend.

  • modelManager.networkPolicy.workloadClusters has the same shape as clusterManager.networkPolicy.workloadClusters (fqdns, cidrs, ports, default 443 and 6443). Empty (the default), it follows cluster-manager's value, since model-manager serves on the clusters cluster-manager composes onto and registers as its backends. Set, it replaces that value as a whole. One helper, agent-platform.modelManager.workloadClusters, resolves it.
  • It renders in model-manager's egress in both flavours whenever model-manager serves through kserve (a static backend, or cluster-manager on), as cluster-manager's does:
    • cilium: toFQDNs for the names and toCIDR for the blocks, on the ports;
    • kubernetes: the blocks, or else every public destination (minus worldExcludedCIDRs), on the ports.
  • Both charts document the value, and the meta chart forwards it.

What is the effect of this change to users?

On an installation that sets clusterManager.networkPolicy.workloadClusters (on AWS the API servers' ELB names, *.*.elb.amazonaws.com), model-manager reaches the same API servers with no new configuration, and a workload-cluster backend answers list_backends. Without a kserve backend, nothing new renders.

How does it look like?

make verify-cluster-manager checks:

  • the cilium egress derives cluster-manager's names and blocks on 443 and 6443;
  • an explicit modelManager.networkPolicy.workloadClusters replaces them (a name on 8443), and cluster-manager's own policy is untouched;
  • nothing renders without a kserve backend;
  • the kubernetes egress follows cluster-manager's workload blocks.

verify-target holds the forwarded empty value out of the golden comparison. make verify-all: 58 of 58.

The local agentlab renders no network policies and has no workload clusters, so the runtime proof is an installation: after the rollout, model-manager's egress there carries the same rule as cluster-manager's, and list_backends with a workload-cluster backend answers healthy.

A kserve backend cluster-manager registers calls a workload cluster's apiserver as the caller; model-manager's egress had no rule for it, so under Cilium list_backends timed out. modelManager.networkPolicy.workloadClusters (the shape of cluster-manager's) follows clusterManager.networkPolicy.workloadClusters when empty and replaces it when set; rendered in both flavours with a kserve backend.
@teemow
teemow requested a review from a team as a code owner September 25, 2026 04:44
@circleci-architect

Copy link
Copy Markdown

Published Helm chart agent-platform-connectivity

4.74.1-rc38233cet20260925044416hb22d6a0
Chart agent-platform-connectivity
Version 4.74.1-rc38233cet20260925044416hb22d6a0
OCI reference oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity:4.74.1-rc38233cet20260925044416hb22d6a0
Digest sha256:2531bc68342bd255976d9df7d553b56d0b2b8d7b21b4d95bddd84bca2b0a6e23
Registry public — gsoci.azurecr.io
Git catalog giantswarm-test-catalog (index)
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity --version 4.74.1-rc38233cet20260925044416hb22d6a0

Posted by architect-orb · build 13933 · commit b22d6a0 · updated in place on every push

@circleci-architect

Copy link
Copy Markdown

Published Helm chart agent-platform

4.74.1-rc38233cet20260925044416hb22d6a0
Chart agent-platform
Version 4.74.1-rc38233cet20260925044416hb22d6a0
OCI reference oci://gsoci.azurecr.io/charts/giantswarm/agent-platform:4.74.1-rc38233cet20260925044416hb22d6a0
Digest sha256:7a713441165f56363f0d908a592b04f4e505402c5b006a6b96a0377ef041219c
Registry public — gsoci.azurecr.io
Git catalog giantswarm-test-catalog (index)
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform --version 4.74.1-rc38233cet20260925044416hb22d6a0

Posted by architect-orb · build 13931 · commit b22d6a0 · updated in place on every push

@teemow
teemow merged commit 866bc0a into main Sep 25, 2026
14 checks passed
@teemow
teemow deleted the model-manager-egress-workload-clusters-687 branch September 25, 2026 05:06
teemow added a commit that referenced this pull request Sep 25, 2026
…en-606

* origin/main: (28 commits)
  feat(dashboards): turn done (final flush) p50 and p95 on Klaus Gateway (#701)
  feat: the managers', the portal's and mcp-kubernetes' traces to the platform's OTLP gateway (#700)
  feat(kagent): compact agent histories only past 600000 tokens (#699)
  fix: keep the single-replica components drainable (maxUnavailable 1) (#697)
  feat(model-serving): export the model pods' traces to the platform's OTLP gateway (#695)
  fix(substrate): keep Substrate's metrics on the scrape path (#694)
  feat: one OTLP collector for every exporter (global.observability.traces.otlp) (#696)
  fix(netpol): admit the scrape of the kagent controller and atenet-egress ext-proc (#691)
  fix(serving): make the models Gateway an internet-facing NLB on AWS (#692)
  fix(model-manager): reach the workload clusters' API servers (#689)
  feat(components): the CloudNativePG operator chart and image from gsoci (#671)
  fix(serving): tear the serving slice down in order, configs freed (#679)
  fix: derive the managers' OAuth login from muster's OAuth server (#678)
  feat(model-serving): stamp the cache claim's tier as annotations (#676)
  fix(substrate): the capz snapshot-store identity holds Storage Blob Delegator on the account (#686)
  fix(connectivity): the egress gateway reaches the Substrate credential provider (#685)
  feat(llm-routing): route the LLM endpoint by model, with API keys (#672)
  feat: the kagent line and the Substrate line at 1.1.0 (#665)
  fix(model-manager): read served models wherever model-manager serves (#681)
  feat(model-serving): check every shipped preset's parsers by model family (#674)
  ...

# Conflicts:
#	CHANGELOG.md
#	tests/verify-gpu-pool.py
teemow added a commit that referenced this pull request Sep 25, 2026
…reset-597

* origin/main: (28 commits)
  feat(dashboards): turn done (final flush) p50 and p95 on Klaus Gateway (#701)
  feat: the managers', the portal's and mcp-kubernetes' traces to the platform's OTLP gateway (#700)
  feat(kagent): compact agent histories only past 600000 tokens (#699)
  fix: keep the single-replica components drainable (maxUnavailable 1) (#697)
  feat(model-serving): export the model pods' traces to the platform's OTLP gateway (#695)
  fix(substrate): keep Substrate's metrics on the scrape path (#694)
  feat: one OTLP collector for every exporter (global.observability.traces.otlp) (#696)
  fix(netpol): admit the scrape of the kagent controller and atenet-egress ext-proc (#691)
  fix(serving): make the models Gateway an internet-facing NLB on AWS (#692)
  fix(model-manager): reach the workload clusters' API servers (#689)
  feat(components): the CloudNativePG operator chart and image from gsoci (#671)
  fix(serving): tear the serving slice down in order, configs freed (#679)
  fix: derive the managers' OAuth login from muster's OAuth server (#678)
  feat(model-serving): stamp the cache claim's tier as annotations (#676)
  fix(substrate): the capz snapshot-store identity holds Storage Blob Delegator on the account (#686)
  fix(connectivity): the egress gateway reaches the Substrate credential provider (#685)
  feat(llm-routing): route the LLM endpoint by model, with API keys (#672)
  feat: the kagent line and the Substrate line at 1.1.0 (#665)
  fix(model-manager): read served models wherever model-manager serves (#681)
  feat(model-serving): check every shipped preset's parsers by model family (#674)
  ...

# Conflicts:
#	CHANGELOG.md
#	tests/verify-gpu-pool.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant