[FEATURE] Add custom CAs for LLM calls

[linsun]

📋 Prerequisites

• I have searched the existing issues to avoid creating a duplicate
• By submitting this issue, you agree to follow our Code of Conduct

📝 Feature Summary

No response

❓ Problem Statement / Motivation

Currently, if a user wants to connect to an internal LLM, which is accessed via TLS certs provided by a CA or intermediate CA, it is not possible to configure it to work with kagent.

Opening this issue to track this request.

💡 Proposed Solution

User should be able to configure the proper TLS certs to access the LLM backend.

🔄 Alternatives Considered

No response

🎯 Affected Service(s)

None

📚 Additional Context

No response

🙋 Are you willing to contribute?

• I am willing to submit a PR for this feature

[tanuj-rai]

Hi @linsun! I'm interested in working on this issue and implementing the feature to support TLS certificates for connecting to an internal LLM via kagent.

Could you please clarify:

1. Where should the TLS settings be added — is there a preferred config file or directory for this?
2. Are there any existing patterns or modules in the project that handle certificate-based configuration that I should follow?

I'm planning to use a configuration like this:

llm_backend:
  url: https://llm.internal.company.com
  tls:
    client_cert: /path/to/client-cert.pem
    client_key: /path/to/client-key.pem
    ca_cert: /path/to/ca-cert.pem

I would really appreciate your guidance. Thanks!

[ArchBlizzard]

Hi @linsun

Progress for this issue so far:

• Added the caCertSecret field to the ModelConfigSpec struct and updated the CRD.
• Updated the controller logic to fetch the CA certificate from the referenced Kubernetes Secret.
• Added support for passing the CA cert to the HTTP client using a new WithCACert option in the client code.
• Wired the CA cert logic into the ModelConfig reconciler.

Current problems:

• Where should the LLM API call logic be placed so it uses the configured client?
• Are there existing tests or testing strategies to validate the custom CA integration with different LLM providers?

Any help or guidance is greatly appreciated - Thank you!

I have also applied for the LFX mentorship and am very interested in contributing to the project.

[inFocus7]

For these secure LLM calls, is this feature request for Declarative agents? Or also for other agent types (e.g. BYO)?

---

kagent-only solution

If solely for Declarative, it should be somewhat straightforward (compared to non-Declarative).

I set up a design below that encapsulates how this would be handled. Just need to work out the details on how exactly we'd safely pass the TLS information (e.g. as a file mount), so the deployed Agent python code can use it. I haven't worked with Python much, but I'd assume we could mount the cert, and then read the file from within the python code & configure the LLM http client(s) with this information.

I think it's basically what ArchBlizzard was suggesting/implementing.

If we also want to support TLS certs for non-Declarative Agents, it may be a bit trickier. We would need some way to (reverse) proxy agent-to-llm traffic from their BYO implementation to use TLS certs. Not sure if/how kagent could handle this without thinking about it more; in this case an actual proxy/gateway may be best.

---

agentgateway-based solution

A more focused solution to the above would be this configuration below. It would require updates.

1. Set up agentgateway with TLS configuration

binds:
- port: 3000
  listeners:
  - routes:
    - policies:
        urlRewrite:
          authority:
            full: api.internal.ai
          path:
            full: "/compatibility/v1/chat/completions"
        backendTLS: {}
        backendAuth:
          key: $API_KEY
      backends:
      - ai:
          name: internal-llm
          hostOverride: api.internal.ai:443
          provider:
            openAI:
              model: command

2. Set up a ModelConfig using agentgateway proxy

apiVersion: kagent.dev/v1alpha2
kind: ModelConfig
metadata:
  name: my-provider-config
  namespace: kagent
spec:
  # I'm not sure if these fields are _required_ for the spec, but the agentgateway would handle these.
  # apiKeySecret: secret
  # apiKeySecretKey: $API_KEY
  # model: command-a-03-2025
  provider: OpenAI
  openAI:
    baseUrl: "https://api.internal.ai/compatibility/v1"
    # this CRD could be "headers: "Host: api.internal.ai", with base url being the proxy to fall more in line with the internal configuration defined below.
    proxyUrl: "agentgateway.ip"

On translation / client configuration, kagent would set up the client (e.g. OpenAI AsyncClient) with the following:

baseUrl: agentgateway.ip
headers:
  Host: https://api.internal.ai/compatibility/v1

Note: From what I saw, the modelConfig already supports defaultHeaders, so this may be used for the Host. I'm going to test and validate this locally.

[inFocus7]

This can be closed with the following PRs merge #1059

[sam-heilbron]

@inFocus7 @jjamroga Given that the linked PR has merged, can this be closed?

Alternatively, are there expansions to the user-facing documentation that ought to be completed to enable end users?

[inFocus7]

@sam-heilbron Yeah it can be closed, since the feature has been implemented + released.

There aren't any user-facing docs. I can work on adding docs; I probably won't be able to work on that until around 2 weeks from now (I'll be out next week). By EOD I can create a docs issue to document this new feature so it's not forgotten 🫡

I am working on user-facing docs for this 🫡: kagent-dev/website#277

KAgent Docs have been updated to show this feature: https://kagent.dev/docs/kagent/supported-providers/byo-openai#tls-configuration

I don't have permissions to close issues, so I didn't close this after it merged