Skip to content

fix(connectivity): catalog ModelConfigs inherit the platform maxTokens - #712

Merged
teemow merged 2 commits into
mainfrom
fix/modelconfig-max-tokens
Sep 26, 2026
Merged

teemow merged 2 commits into
mainfrom
fix/modelconfig-max-tokens

Conversation

@teemow

@teemow teemow commented Sep 26, 2026

Copy link
Copy Markdown
Member

Closes #702.

Problem

A kagent.modelConfigs[] entry of the connectivity chart renders no maxTokens, so an agent on a catalog model (anthropic-opus-5, anthropic-opus-5-5) stops a long answer at kagent's built-in 8192 tokens, thinking included. The default ModelConfig has 32000: the platform manager sets kagent.providers.anthropic.config.maxTokens for every installation (giantswarm/giantswarm-platform-manager#206), and the connectivity release already receives that value; only the default ModelConfig used it.

Change

  • An Anthropic entry inherits kagent.providers.anthropic.config.maxTokens the way it already inherits promptCaching/cacheTTL, in the same provider block next to the listener baseUrl.
  • An entry's own maxTokens wins and lands in its provider's block (Anthropic, AnthropicVertexAI, AzureOpenAI, Mistral, OpenAI: the ModelConfigSpec blocks that have the field). On any other provider the render fails naming the entry, instead of the API server pruning it silently.
  • The chart alone invents no default.
  • make verify-max-tokens asserts inheritance, precedence, the no-default case and the guard. verify-kagent-crds validates an entry with maxTokens against the kagent line's CRD.

No values change is needed on any installation: the four Opus agents get 32000 on the next rollout.

Verification

  • make verify-max-tokens verify-prompt-caching verify-llm-routing verify-kagent-crds pass locally.
  • After release: the Opus ModelConfigs report spec.anthropic.maxTokens: 32000, and a long Opus turn runs past 8192 tokens.

@teemow
teemow requested a review from a team as a code owner September 26, 2026 07:52
@circleci-architect

circleci-architect Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Published Helm chart agent-platform

4.84.1-r79a95826t20260926153617ha980ae9
Chart agent-platform
Version 4.84.1-r79a95826t20260926153617ha980ae9
OCI reference oci://gsoci.azurecr.io/charts/giantswarm/agent-platform:4.84.1-r79a95826t20260926153617ha980ae9
Digest sha256:90d8a01e01b2318d5aec4759b4994c49eba048ba66c94ca04c7678df768e1a1d
Registry public — gsoci.azurecr.io
Git catalog giantswarm-test-catalog (index)
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform --version 4.84.1-r79a95826t20260926153617ha980ae9

Posted by architect-orb · build 14938 · commit a980ae9 · updated in place on every push

@circleci-architect

circleci-architect Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Published Helm chart agent-platform-connectivity

4.84.1-r79a95826t20260926153617ha980ae9
Chart agent-platform-connectivity
Version 4.84.1-r79a95826t20260926153617ha980ae9
OCI reference oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity:4.84.1-r79a95826t20260926153617ha980ae9
Digest sha256:5ca6ea2dbe54519752f1441f5bc788841a9e6d0bf35e04d82d90563918d97a15
Registry public — gsoci.azurecr.io
Git catalog giantswarm-test-catalog (index)
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity --version 4.84.1-r79a95826t20260926153617ha980ae9

Posted by architect-orb · build 14937 · commit a980ae9 · updated in place on every push

@teemow

teemow commented Sep 26, 2026

Copy link
Copy Markdown
Member Author

From Timo's agent: agentlab proof on this branch (platform.chartPath, both charts from the checkout; the connectivity chart pushed to the lab registry as sha256:31ee3b5b and run by HelmRelease agent-platform-connectivity 1.1.35+31ee3b5be0c0). The values overlay mirrors an installation: kagent.providers.anthropic.config.maxTokens: 32000 plus one catalog entry anthropic-opus-5 without its own bound.

NAME                   MODEL               MAXTOK   CACHE
anthropic-opus-5       claude-opus-5       32000    true
default-model-config   claude-sonnet-4-6   32000    true

The API server admitted the inherited maxTokens next to the inherited promptCaching (not pruned). agentlab platform-test passes.

…okens

* origin/main:
  feat(connectivity): retire the model-manager REST route (#711)
  feat(model-serving): gemma-4-31b at 32k with an fp8 KV cache (#709)
  feat(model-serving): the muse-glimmer-30b preset on vLLM's own image (#673)
  fix(model-serving): gpt-oss reads its tiktoken encodings from the image (#675)
  fix(hooks): serving teardown retries once Flux's API is gone (#707)
  feat(cluster-manager): provider aws opens the CAPA API server LBs (#704)

# Conflicts:
#	CHANGELOG.md
@teemow
teemow merged commit 00ea5d7 into main Sep 26, 2026
13 checks passed
@teemow
teemow deleted the fix/modelconfig-max-tokens branch September 26, 2026 15:57
teemow added a commit that referenced this pull request Sep 26, 2026
…n-post-delete

* origin/main:
  fix(connectivity): catalog ModelConfigs inherit the platform maxTokens (#712)
  feat(connectivity): retire the model-manager REST route (#711)
  feat(model-serving): gemma-4-31b at 32k with an fp8 KV cache (#709)
  feat(model-serving): the muse-glimmer-30b preset on vLLM's own image (#673)
  fix(model-serving): gpt-oss reads its tiktoken encodings from the image (#675)
  fix(hooks): serving teardown retries once Flux's API is gone (#707)
  feat(cluster-manager): provider aws opens the CAPA API server LBs (#704)

Signed-off-by: Timo Derstappen <teemow@gmail.com>

# Conflicts:
#	CHANGELOG.md
teemow added a commit that referenced this pull request Sep 26, 2026
…se-placement

* origin/main:
  fix(hooks): the bundled engine goes after Helm's delete pass (#713)
  fix(connectivity): catalog ModelConfigs inherit the platform maxTokens (#712)
  feat(connectivity): retire the model-manager REST route (#711)
  feat(model-serving): gemma-4-31b at 32k with an fp8 KV cache (#709)
  feat(model-serving): the muse-glimmer-30b preset on vLLM's own image (#673)

# Conflicts:
#	CHANGELOG.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gazelle: three Opus agents' ModelConfig has no maxTokens, output caps at kagent's 8192 default

1 participant