Fix Session Affinity With Partitioned Session Cookies

[hoffmaen]

Current behavior

There are currently two issues/bugs/unexpected behaviors with Partitioned Session Cookies in Gorouter.

The issues described here originate from the fact that CHIPS (Cookies Having Independent Partitioned State) introduces a new dimension to cookies (see RFC6265bis), allowing multiple cookies with the same name, domain, and path but different partitioned attributes to co-exist. The cookie attributes reside on the client side and are not sent to the server.

0. Session Affinity a/k/a Sticky Sessions - Green Path

The following diagram shows how session affinity in Gorouter works in generel:

1. Defaulting to Issue Non-Partitioned Cookie

Gorouter sets an unpartitioned VCAP_ID cookie when the provided one points to a non-existent instance and no new JSESSIONID cookie was issued by the application.

Context

This issue arises when an application instance sets a partitioned JSESSIONID cookie in the response (causing Gorouter to add a partitioned VCAP_ID cookie), and subsequently, this application instance becomes unavailable.

The next request (containing partitioned VCAP_ID and partitioned JSESSIONID cookies) reaches Gorouter, but Gorouter cannot route to the instance referenced by the VCAP_ID cookie. In this case, Gorouter's default behavior is to route the request to another available instance. When that instance responds without a JSESSIONID cookie, Gorouter will always set a new unpartitioned VCAP_ID cookie.

Problem

This leaves the client with:

• A partitioned JSESSIONID cookie
• A partitioned VCAP_ID cookie referencing a non-existent app instance
• An unpartitioned VCAP_ID cookie referencing an existing instance

This creates problems in two scenarios:

• Non-Third-Party Context: The client sends all three cookies to Gorouter. Gorouter will either pick the partitioned VCAP_ID cookie first (routing the request to an arbitrary instance) or pick the unpartitioned cookie first (routing it to the instance referenced in that cookie).
• Third-Party Context: Cookie sandboxing is broken because the new VCAP_ID does not have the partitioned attribute, causing it to be sent to any host embedding the application. Additionally, the client sends both the (orphaned) partitioned and unpartitioned VCAP_ID cookies, and Gorouter will prefer one depending on client behavior.

Solution

Gorouter defaults to creating a non-partitioned VCAP_ID cookie in this case because cookie attributes are not part of the cookie header sent by the client. Gorouter does not know whether the client-side VCAP_ID cookie referencing an invalid instance has the partitioned flag set, and therefore cannot create a new cookie with the same attributes as the old one.

Proposal: Gorouter should not create new VCAP_ID cookies when the referenced instance is invalid and the application does not set a new JSESSIONID cookie. This might break existing scenarios.

2. Gorouter Only Translates the First JSESSIONID Cookie to a VCAP_ID Cookie

Context

Google describes a migration strategy for CHIPS that addresses the issue of partitioned and non-partitioned cookies co-existing. They propose the following migration approach: When migration begins, the application sets two cookies:

• A partitioned cookie
• An unpartitioned cookie with Max-Age=0, which immediately invalidates the cookie

Problem

Applications on Cloud Foundry cannot implement this strategy because it interferes with the VCAP_ID cookie set by Gorouter.

Gorouter currently only parses the first JSESSIONID cookie and creates a VCAP_ID cookie based on it. Then, Gorouter breaks out of the loop.

For correct implementation, when an application sets both JSESSIONID cookies as described above, Gorouter needs to "translate" both cookies to VCAP_ID cookies and copy the attributes (Partitioned state, Max-Age) from the JSESSIONID cookies to each VCAP_ID cookie.

Solution

Gorouter must support the proposed migration strategy.

Proposal: Gorouter should not break out of the loop when iterating through response cookies. Instead, it should create one VCAP_ID cookie per JSESSIONID cookie and copy all attributes, even when there are multiple JSESSIONID cookies.

Affected Version

current

[maxmoehl]

The current logic around preventing multiple VCAP_ID cookies being set also seems flawed:

routing-release/src/code.cloudfoundry.org/gorouter/proxy/round_tripper/proxy_round_tripper.go

Lines 521 to 526 in f6a73cb

	for _, v := range response.Cookies() {
	if v.Name == VcapCookieId {
	shouldSetVCAPID = false
	break
	}
	}

It only considers the Set-Cookie headers it received from the app, but the client might already have a VCAP_ID cookie that it sent in its Cookie header. We can't really prevent multiple VCAP_ID cookies being set on the client and I agree that we should remove that block and fully mirror the apps sticky session cookies, even sending multiple Set-Cookie headers if that's what the app is doing with its sticky session cookies.

On a side-note, if the app happens to use the VCAP_ID cookie for something else, gorouter stickiness just does not work at all, but I think that's ok.

[hoffmaen]

Added some sequence diagram that show

Default Behavior:

Problem 1: Unpartitioned VCAP_ID cookie set

Problem 2: Ignoring multiple JSESSIONID cookies

[sergioRozenszajn]

This is a breaking change. Session stickiness is a hard requirement for correct approuter session handling and cannot be disabled
or lost.

Without Redis:
If session stickiness is lost and a request with the same JSESSIONID cookie arrives at a different approuter instance where no
session exists, the user will be forced to re-authenticate, causing work disruption.

With Redis (Still Problematic):
Even with Redis-backed session storage, losing session stickiness causes data inconsistency:

1. Request arrives at Instance B (session not in memory)
2. Instance B retrieves the session from Redis
3. Instance B updates the session (e.g., adds backend cookies)
4. Next request arrives at Instance A (original instance)
5. Instance A uses its stale in-memory session (without the updates from Instance B)
6. Result: Session data loss and broken backend connectivity

Fixing:
When Redis is available, we can detect the situation where the VCAP_ID cookie is missing when fetching a user session from Redis,
and generate a new JSESSIONID cookie to re-establish session stickiness. However, customers without Redis will still be forced to
log in again, causing work disruption.

Conclusion:
Session stickiness must be maintained. Redis provides session recovery after instance restarts and enables automatic stickiness
re-establishment, but does not eliminate the requirement for session stickiness during normal operation. Without sticky sessions,
even Redis-backed deployments will experience session inconsistency and authentication failures.