Repository navigation
fix(model-manager): reach the workload clusters' API servers - #689
Merged
Merged
Conversation
A kserve backend cluster-manager registers calls a workload cluster's apiserver as the caller; model-manager's egress had no rule for it, so under Cilium list_backends timed out. modelManager.networkPolicy.workloadClusters (the shape of cluster-manager's) follows clusterManager.networkPolicy.workloadClusters when empty and replaces it when set; rendered in both flavours with a kserve backend.
Published Helm chart
|
| Chart | agent-platform-connectivity |
| Version | 4.74.1-rc38233cet20260925044416hb22d6a0 |
| OCI reference | oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity:4.74.1-rc38233cet20260925044416hb22d6a0 |
| Digest | sha256:2531bc68342bd255976d9df7d553b56d0b2b8d7b21b4d95bddd84bca2b0a6e23 |
| Registry | public — gsoci.azurecr.io |
| Git catalog | giantswarm-test-catalog (index) |
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity --version 4.74.1-rc38233cet20260925044416hb22d6a0Posted by architect-orb · build 13933 · commit b22d6a0 · updated in place on every push
Published Helm chart
|
| Chart | agent-platform |
| Version | 4.74.1-rc38233cet20260925044416hb22d6a0 |
| OCI reference | oci://gsoci.azurecr.io/charts/giantswarm/agent-platform:4.74.1-rc38233cet20260925044416hb22d6a0 |
| Digest | sha256:7a713441165f56363f0d908a592b04f4e505402c5b006a6b96a0377ef041219c |
| Registry | public — gsoci.azurecr.io |
| Git catalog | giantswarm-test-catalog (index) |
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform --version 4.74.1-rc38233cet20260925044416hb22d6a0Posted by architect-orb · build 13931 · commit b22d6a0 · updated in place on every push
3 tasks done
teemow
added a commit
that referenced
this pull request
Sep 25, 2026
…en-606 * origin/main: (28 commits) feat(dashboards): turn done (final flush) p50 and p95 on Klaus Gateway (#701) feat: the managers', the portal's and mcp-kubernetes' traces to the platform's OTLP gateway (#700) feat(kagent): compact agent histories only past 600000 tokens (#699) fix: keep the single-replica components drainable (maxUnavailable 1) (#697) feat(model-serving): export the model pods' traces to the platform's OTLP gateway (#695) fix(substrate): keep Substrate's metrics on the scrape path (#694) feat: one OTLP collector for every exporter (global.observability.traces.otlp) (#696) fix(netpol): admit the scrape of the kagent controller and atenet-egress ext-proc (#691) fix(serving): make the models Gateway an internet-facing NLB on AWS (#692) fix(model-manager): reach the workload clusters' API servers (#689) feat(components): the CloudNativePG operator chart and image from gsoci (#671) fix(serving): tear the serving slice down in order, configs freed (#679) fix: derive the managers' OAuth login from muster's OAuth server (#678) feat(model-serving): stamp the cache claim's tier as annotations (#676) fix(substrate): the capz snapshot-store identity holds Storage Blob Delegator on the account (#686) fix(connectivity): the egress gateway reaches the Substrate credential provider (#685) feat(llm-routing): route the LLM endpoint by model, with API keys (#672) feat: the kagent line and the Substrate line at 1.1.0 (#665) fix(model-manager): read served models wherever model-manager serves (#681) feat(model-serving): check every shipped preset's parsers by model family (#674) ... # Conflicts: # CHANGELOG.md # tests/verify-gpu-pool.py
teemow
added a commit
that referenced
this pull request
Sep 25, 2026
…reset-597 * origin/main: (28 commits) feat(dashboards): turn done (final flush) p50 and p95 on Klaus Gateway (#701) feat: the managers', the portal's and mcp-kubernetes' traces to the platform's OTLP gateway (#700) feat(kagent): compact agent histories only past 600000 tokens (#699) fix: keep the single-replica components drainable (maxUnavailable 1) (#697) feat(model-serving): export the model pods' traces to the platform's OTLP gateway (#695) fix(substrate): keep Substrate's metrics on the scrape path (#694) feat: one OTLP collector for every exporter (global.observability.traces.otlp) (#696) fix(netpol): admit the scrape of the kagent controller and atenet-egress ext-proc (#691) fix(serving): make the models Gateway an internet-facing NLB on AWS (#692) fix(model-manager): reach the workload clusters' API servers (#689) feat(components): the CloudNativePG operator chart and image from gsoci (#671) fix(serving): tear the serving slice down in order, configs freed (#679) fix: derive the managers' OAuth login from muster's OAuth server (#678) feat(model-serving): stamp the cache claim's tier as annotations (#676) fix(substrate): the capz snapshot-store identity holds Storage Blob Delegator on the account (#686) fix(connectivity): the egress gateway reaches the Substrate credential provider (#685) feat(llm-routing): route the LLM endpoint by model, with API keys (#672) feat: the kagent line and the Substrate line at 1.1.0 (#665) fix(model-manager): read served models wherever model-manager serves (#681) feat(model-serving): check every shipped preset's parsers by model family (#674) ... # Conflicts: # CHANGELOG.md # tests/verify-gpu-pool.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Towards #687 (its live
list_backendscriterion waits for the next GPU window). A kserve backend that cluster-manager registers targets a workload cluster's apiserver, which model-manager calls as the caller. model-manager's egress policy had no rule to those API servers, so under the cilium flavourlist_backendstimed out dialing the cluster's API server (dial tcp <ELB>:443: i/o timeout), and so did every tool naming that backend.modelManager.networkPolicy.workloadClustershas the same shape asclusterManager.networkPolicy.workloadClusters(fqdns,cidrs,ports, default 443 and 6443). Empty (the default), it follows cluster-manager's value, since model-manager serves on the clusters cluster-manager composes onto and registers as its backends. Set, it replaces that value as a whole. One helper,agent-platform.modelManager.workloadClusters, resolves it.toFQDNsfor the names andtoCIDRfor the blocks, on the ports;worldExcludedCIDRs), on the ports.What is the effect of this change to users?
On an installation that sets
clusterManager.networkPolicy.workloadClusters(on AWS the API servers' ELB names,*.*.elb.amazonaws.com), model-manager reaches the same API servers with no new configuration, and a workload-cluster backend answerslist_backends. Without a kserve backend, nothing new renders.How does it look like?
make verify-cluster-managerchecks:modelManager.networkPolicy.workloadClustersreplaces them (a name on 8443), and cluster-manager's own policy is untouched;verify-targetholds the forwarded empty value out of the golden comparison.make verify-all: 58 of 58.The local agentlab renders no network policies and has no workload clusters, so the runtime proof is an installation: after the rollout, model-manager's egress there carries the same rule as cluster-manager's, and
list_backendswith a workload-cluster backend answers healthy.