RFC: App-to-App mTLS via GoRouter

[rkoster]

Summary

This RFC proposes enabling authenticated and authorized app-to-app communication via GoRouter using mutual TLS (mTLS).

View the full RFC

Applications connect to a shared internal domain (apps.mtls.internal), where GoRouter:

• Requires and validates client certificates (Instance Identity)
• Enforces per-route access control using allowed_sources
• Follows a default-deny model (aligned with C2C network policies)

Key Points

• No new infrastructure: Uses existing GoRouter with domain-specific mTLS configuration
• Default-deny security: Routes blocked unless explicitly allowed via allowed_sources
• Internal-only domain: apps.mtls.internal is a separate domain tree, avoiding conflicts with existing apps.internal routes
• Depends on RFC-0027: Route options framework required for allowed_sources support

Implementation Phases

• Phase 1a (mTLS Infrastructure): GoRouter validates client certificates for the mTLS domain
• Phase 1b (Authorization Enforcement): Per-route access control via allowed_sources (co-requisite with 1a)
• Phase 2 (Optional): Egress HTTP proxy on port 8888 for simplified client adoption

cc @cloudfoundry/toc @cloudfoundry/wg-app-runtime-interfaces

[silvestre]

I really like this proposal.

Just to be sure: It would be still possible to have an apps.mtls.internal route allowing access for any source app, so that the authorization check could be done in the app, right?

One use-case would be in the app-autoscaler service, where we expose an mTLS endpoint but check the authorization by determining if the app is bound to an autoscaler service instance, which is dynamic information we could not determine during route creation.

[rkoster]

I really like this proposal.

Just to be sure: It would be still possible to have an apps.mtls.internal route allowing access for any source app, so that the authorization check could be done in the app, right?

One use-case would be in the app-autoscaler service, where we expose an mTLS endpoint but check the authorization by determining if the app is bound to an autoscaler service instance, which is dynamic information we could not determine during route creation.

@silvestre I have update the RFC with:

applications:
- name: autoscaler-api
  routes:
  - route: autoscaler.apps.mtls.internal
    options:
      allowed_sources:
        any: true

[theghost5800]

This idea is really interesting but will be possible to have communication to app containers on different ports or different protocol than http?

[rkoster]

This idea is really interesting but will be possible to have communication to app containers on different ports or different protocol than http?

This RFC currently focuses on HTTP traffic via GoRouter, but non-HTTP protocol support is an interesting future direction.

Current constraints:

• GoRouter uses Go's httputil.ReverseProxy which handles HTTP semantics (headers, paths, etc.)
• Caller identity is forwarded via the XFCC HTTP header, which doesn't exist for raw TCP
• GoRouter does not currently support HTTP CONNECT method for tunneling

What would be needed for non-HTTP support:

1. HTTP CONNECT tunneling in GoRouter: GoRouter would need to detect CONNECT requests, validate mTLS + allowed_sources, then hijack the connection and relay raw TCP bytes to the backend. The pattern exists (similar to WebSocket upgrades), but would require new implementation.
2. Identity forwarding challenge: Inside a TCP tunnel there's no XFCC header. Options include:

• PROXY protocol v2 (GoRouter sends client cert info as TLV before the TCP stream)
• Backend also requires mTLS and validates the client cert directly
• Application-layer identity (less secure)

3. Envoy egress proxy: The Phase 2 egress proxy (Envoy) already supports HTTP CONNECT tunneling, so apps could potentially use CONNECT backend.apps.mtls.internal:5432 to tunnel arbitrary protocols. But GoRouter still needs to support CONNECT for this to work end-to-end.

For now, this is out of scope to keep the RFC focused and achievable. But feel free to create a follow up RFC for Non-HTTP use cases.

[maxmoehl]

We need to be careful with what we add to the per-route features. Each and every option we set in there will be transmitted via the NATS bus every 20s (unless adjusted by the operator) from each app instance to each gorouter instance. Even a slight increase in message size can have quite noticeable effects on the overall bandwidth consumption.

One of the thoughts we had, is to only allow very simple rules. Specifically allow from instances of the same app, from all apps within a space, from all apps within an org. This could be a simple enum option allowed_scope: app / space / org (name is just for illustration purposes) which would keep the size within a predictable scope.

If we go for a more flexible option we must introduce strict limits on the number of bytes each user is allowed to add to each route.

[maxmoehl]

Especially for larger deployments it can be beneficial to have a TCP load balancer in front of gorouter to guarantee a reliable load balancing across gorouter instances as DNS load balancing is always at the mercy of a good client-side implementation.

I don't see why this wouldn't be possible with the proposed design, just wanted to mention it.