Traffic shifting with envoy.extensions.filters.http.stateful_session.v3.StatefulSession

[pcj]

Summary

I'm trying to get the following scenario to work (copied from a slack thread):

Hello, does istio support traffic splitting and session affinity (at the same time)? For example, if a user loads an html page for /login, it will be randomized to either app-v1 or app-v2. However, once that choice has been made, the login page .js files and .css files should either come from app-v1 or app-v2 , but not both (for a particular session)

This is an important invariant to hold when doing blue/green or canary deployments, because the javascript and css files are likely only internally consistent between app versions. It does not make sense to serve half the application from v1 and the other half from v2.

I am trying to demonstrate this using the Traffic Shifting example, hoping to use the feature developed in #18207 to satisfy this requirement.

The TL;DR; is that while I can demonstrate traffic shifting and the stateful session cookie being set, the presence of the cookie does not seem to override the routing decision make by the traffic shift.

Repro

$ k version --short
Client Version: v1.21.1
Server Version: v1.19.2

$ TAG=$(curl https://storage.googleapis.com/istio-build/dev/latest)
1.14-alpha.1ea132c507ad870c7daef1c79dc6cbe5d328bc15
$ curl -OJL https://storage.googleapis.com/istio-build/dev/$TAG/istioctl-$TAG-osx.tar.gz
$ tar xvzf istioctl-1.14-alpha.1ea132c507ad870c7daef1c79dc6cbe5d328bc15-osx.tar.gz
$ mv ./istioctl ~/bin/
$ istioctl version
no running Istio pods in "istio-system"
1.14-alpha.1ea132c507ad870c7daef1c79dc6cbe5d328bc15
$ istioctl install --set tag=$TAG --set hub=gcr.io/istio-testing --dry-run

As it turns out, the master version did not have #18207 fully merged in. istio/proxy#3689 should fix that. As a workaround, I am using gcr.io/howardjohn-istio/proxyv2:stateful2 and specifically sha256:47df8c1c46f5961c57919969454664d4e30887f46893a34fa1dc1b62f849716f.

Bookinfo

$ k create ns bookinfo
$ kubectl label namespace bookinfo istio-injection=enabled
$ k apply bookinfo.yaml

# Copyright Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

##################################################################################################
# This file defines the services, service accounts, and deployments for the Bookinfo sample.
#
# To apply all 4 Bookinfo services, their corresponding service accounts, and deployments:
#
#   kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
#
# Alternatively, you can deploy any resource separately:
#
#   kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml -l service=reviews # reviews Service
#   kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml -l account=reviews # reviews ServiceAccount
#   kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml -l app=reviews,version=v3 # reviews-v3 Deployment
##################################################################################################

##################################################################################################
# Details service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: details
  namespace: bookinfo
  labels:
    app: details
    service: details
spec:
  ports:
    - port: 9080
      name: http
  selector:
    app: details
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-details
  namespace: bookinfo
  labels:
    account: details
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: details-v1
  namespace: bookinfo
  labels:
    app: details
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: details
      version: v1
  template:
    metadata:
      labels:
        app: details
        version: v1
    spec:
      serviceAccountName: bookinfo-details
      containers:
        - name: details
          image: docker.io/istio/examples-bookinfo-details-v1:1.16.2
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 9080
          securityContext:
            runAsUser: 1000
        - name: istio-proxy
          image: gcr.io/howardjohn-istio/proxyv2:stateful2
          imagePullPolicy: IfNotPresent

---
##################################################################################################
# Ratings service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: ratings
  namespace: bookinfo
  labels:
    app: ratings
    service: ratings
spec:
  ports:
    - port: 9080
      name: http
  selector:
    app: ratings
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-ratings
  namespace: bookinfo
  labels:
    account: ratings
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratings-v1
  namespace: bookinfo
  labels:
    app: ratings
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratings
      version: v1
  template:
    metadata:
      labels:
        app: ratings
        version: v1
    spec:
      serviceAccountName: bookinfo-ratings
      containers:
        - name: ratings
          image: docker.io/istio/examples-bookinfo-ratings-v1:1.16.2
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 9080
          securityContext:
            runAsUser: 1000
        - name: istio-proxy
          image: gcr.io/howardjohn-istio/proxyv2:stateful2
          imagePullPolicy: IfNotPresent

---
##################################################################################################
# Reviews service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: reviews
  namespace: bookinfo
  labels:
    app: reviews
    service: reviews
spec:
  ports:
    - port: 9080
      name: http
  selector:
    app: reviews
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-reviews
  namespace: bookinfo
  labels:
    account: reviews
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v1
  namespace: bookinfo
  labels:
    app: reviews
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v1
  template:
    metadata:
      labels:
        app: reviews
        version: v1
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
        - name: istio-proxy
          image: gcr.io/howardjohn-istio/proxyv2:stateful2
          imagePullPolicy: IfNotPresent
        - name: reviews
          image: docker.io/istio/examples-bookinfo-reviews-v1:1.16.2
          imagePullPolicy: IfNotPresent
          env:
            - name: LOG_DIR
              value: "/tmp/logs"
          ports:
            - containerPort: 9080
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
          securityContext:
            runAsUser: 1000
      volumes:
        - name: wlp-output
          emptyDir: {}
        - name: tmp
          emptyDir: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v2
  namespace: bookinfo
  labels:
    app: reviews
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v2
  template:
    metadata:
      labels:
        app: reviews
        version: v2
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
        - name: istio-proxy
          image: gcr.io/howardjohn-istio/proxyv2:stateful2
          imagePullPolicy: IfNotPresent
        - name: reviews
          image: docker.io/istio/examples-bookinfo-reviews-v2:1.16.2
          imagePullPolicy: IfNotPresent
          env:
            - name: LOG_DIR
              value: "/tmp/logs"
          ports:
            - containerPort: 9080
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
          securityContext:
            runAsUser: 1000
      volumes:
        - name: wlp-output
          emptyDir: {}
        - name: tmp
          emptyDir: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v3
  namespace: bookinfo
  labels:
    app: reviews
    version: v3
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v3
  template:
    metadata:
      labels:
        app: reviews
        version: v3
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
        - name: istio-proxy
          image: gcr.io/howardjohn-istio/proxyv2:stateful2
          imagePullPolicy: IfNotPresent
        - name: reviews
          image: docker.io/istio/examples-bookinfo-reviews-v3:1.16.2
          imagePullPolicy: IfNotPresent
          env:
            - name: LOG_DIR
              value: "/tmp/logs"
          ports:
            - containerPort: 9080
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
          securityContext:
            runAsUser: 1000
      volumes:
        - name: wlp-output
          emptyDir: {}
        - name: tmp
          emptyDir: {}
---
##################################################################################################
# Productpage services
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: productpage
  namespace: bookinfo
  labels:
    app: productpage
    service: productpage
spec:
  ports:
    - port: 9080
      name: http
  selector:
    app: productpage
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-productpage
  namespace: bookinfo
  labels:
    account: productpage
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: productpage-v1
  namespace: bookinfo
  labels:
    app: productpage
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: productpage
      version: v1
  template:
    metadata:
      labels:
        app: productpage
        version: v1
    spec:
      serviceAccountName: bookinfo-productpage
      containers:
        - name: istio-proxy
          image: gcr.io/howardjohn-istio/proxyv2:stateful2
          imagePullPolicy: IfNotPresent
        - name: productpage
          image: docker.io/istio/examples-bookinfo-productpage-v1:1.16.2
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 9080
          volumeMounts:
            - name: tmp
              mountPath: /tmp
          securityContext:
            runAsUser: 1000
      volumes:
        - name: tmp
          emptyDir: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: productpage-v2
  namespace: bookinfo
  labels:
    app: productpage
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: productpage
      version: v2
  template:
    metadata:
      labels:
        app: productpage
        version: v2
    spec:
      serviceAccountName: bookinfo-productpage
      containers:
        - name: istio-proxy
          image: gcr.io/howardjohn-istio/proxyv2:stateful2
          imagePullPolicy: IfNotPresent
        - name: productpage
          image: docker.io/istio/examples-bookinfo-productpage-v1:1.16.2
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 9080
          volumeMounts:
            - name: tmp
              mountPath: /tmp
          securityContext:
            runAsUser: 1000
      volumes:
        - name: tmp
          emptyDir: {}
---

$ k apply bookinfo-gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
  namespace: bookinfo
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "*"

$ k apply virtual-service-all-v1.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: productpage
  namespace: bookinfo
spec:
  gateways:
    - bookinfo-gateway
  hosts:
    - "*"
  http:
    - route:
        - destination:
            host: productpage
            subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
  namespace: bookinfo
spec:
  hosts:
    - reviews
  http:
    - route:
        - destination:
            host: reviews
            subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
  namespace: bookinfo
spec:
  hosts:
    - ratings
  http:
    - route:
        - destination:
            host: ratings
            subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: details
  namespace: bookinfo
spec:
  hosts:
    - details
  http:
    - route:
        - destination:
            host: details
            subset: v1
---

Also:

apiVersion: telemetry.istio.io/v1alpha1
kind: Telemetry
metadata:
  name: mesh-bookinfo
  namespace: bookinfo
spec:
  accessLogging:
    - providers:
        - name: envoy

At this point the bookinfo app is working:

$ curl -s -D - -o /dev/null http://20.104.235.95/productpage
HTTP/1.1 200 OK
content-type: text/html; charset=utf-8
content-length: 4183
server: istio-envoy
date: Tue, 01 Feb 2022 23:08:05 GMT
x-envoy-upstream-service-time: 30

Applying a 50-50 split per the example:

$ k apply virtual-service-reviews-50-v3.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
  namespace: bookinfo
spec:
  hosts:
    - reviews
  http:
    - route:
        - destination:
            host: reviews
            subset: v1
          weight: 50
        - destination:
            host: reviews
            subset: v3
          weight: 50

This turned out not to work, so I also created a productpage-v2 and a 50/50 split for that for experimentation:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: productpage
  namespace: bookinfo
spec:
  gateways:
    - bookinfo-gateway
  hosts:
    - "*"
  http:
    - route:
        - destination:
            host: productpage
            subset: v1
          weight: 50
        - destination:
            host: productpage
            subset: v2
          weight: 50

Stateful Session

I used various permutations of the envoyfilters, but these are the two I ended up experimenting with the most:

$ k apply stateful-gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: stateful
  namespace: istio-system
spec:
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.stateful_session
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.stateful_session.v3.StatefulSession
            session_state:
              name: envoy.http.stateful_session.cookie
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.http.stateful_session.cookie.v3.CookieBasedSessionState
                cookie:
                  # Name of the cookie
                  name: gateway-session-cookie
                  # Path to set. Exercise caution here, as https://github.com/envoyproxy/envoy/issues/19655 may break persistence
                  path: /
                  # TTL of cookie. Can be set to 0 for session based
                  # ttl: 120s

$ k apply stateful-productpage.yaml

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: stateful
  namespace: bookinfo
spec:
  workloadSelector:
    labels:
      app: productpage
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          # portNumber: 9081
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.stateful_session
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.stateful_session.v3.StatefulSession
            session_state:
              name: envoy.http.stateful_session.cookie
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.http.stateful_session.cookie.v3.CookieBasedSessionState
                cookie:
                  # Name of the cookie
                  name: bookinfo-session-cookie
                  # Path to set. Exercise caution here, as https://github.com/envoyproxy/envoy/issues/19655 may break persistence
                  path: /
                  # TTL of cookie. Can be set to 0 for session based
                  # ttl: "0"

Having applied the stateful-gateway (for example), the cookie is indeed being set :

curl -s -D - -o /dev/null http://20.104.235.95/productpage
$ HTTP/1.1 200 OK
content-type: text/html; charset=utf-8
content-length: 4183
server: istio-envoy
date: Tue, 01 Feb 2022 23:08:05 GMT
x-envoy-upstream-service-time: 30
set-cookie: gateway-session-cookie="MTAuMjQ0LjEuMTA1OjkwODA="; Path=/; HttpOnly

However, this was not effective in making the traffic route to productpage-v1 or productpage-v2 reliably... I still observed a similar 50/50 distribution of the traffic split.

Other experiments using different contexts (SIDECAR_OUTBOUND), different workload selectors (e.g., for reviews), and other permutations either had the same outcome (no stickyness), or no cookie being set on the final response (curl or browser).

Hoping to get this working! Thanks in advance.

cc @wbpcode @howardjohn

[pcj]

If this issue is more appropriate for istio, happy to move it.

[howardjohn]

Just to double check - on subsequent curl requests did you send the cookie? Curl doesn't automatically. I think you mentioned browser which should do that automatically but just checking

[pcj]

Good point, on subsequent curl requests I did forget to do that, but I was mostly using the browser for testing, but copying the curl output for the issue. So I think the issue is valid (but I will go double-check this).

[pcj]

Request 1:

Request 2:

[pcj]

As an aside, the browser requests for css sourcemaps seemed to have a negative affinity.

[htuch]

CC @wbpcode @dio

[wbpcode]

/assign

[wbpcode]

hi, @pcj Traffic Shifting is implemented by the weighted clusters in the Envoy. Envoy will choose a cluster randomly according to the cluster weight before the load balancing. So both hash-based session stickiness and stateful session stickiness are not worked for this scenario.

[pcj]

@wbpcode Thank you for looking into this. Is the conclusion that it is not possible to implement this scenario?

[pcj]

FWIW (prior art), this turns out to be supported by apache (both session stickyness and traffic shifting). A little sad that envoy can not 😢

<Proxy balancer://default-accstorefront-http2>
    # endpoint weight: 25
    BalancerMember  http://10.244.18.8:8081  loadfactor=25   keepalive=On  route=accstorefront-20220117-2-76b86667ff-ghb82
    # endpoint weight: 25
    BalancerMember  http://10.244.25.2:8081  loadfactor=25   keepalive=On  route=accstorefront-20220117-2-76b86667ff-fvl47
    # endpoint weight: 25
    BalancerMember  http://10.244.25.3:8081  loadfactor=25   keepalive=On  route=accstorefront-20220117-2-76b86667ff-p5kcn
    # endpoint weight: 25
    BalancerMember  http://10.244.7.41:8081  loadfactor=25   keepalive=On  route=accstorefront-20220117-2-76b86667ff-s9bsf

    Header add Set-Cookie "ROUTE=.%{BALANCER_WORKER_ROUTE}e; Path=/; Secure; HttpOnly" env=BALANCER_ROUTE_CHANGED
    ProxySet   stickysession=ROUTE
</Proxy>

[wbpcode]

Is the conclusion that it is not possible to implement this scenario?

Envoy can worked for this scenario, but Istio cannot. It's possible for Envoy to implemented Traffic Shifting with LB with weight. Then both hash-based session stickiness and stateful session stickiness will worked properly.

But Istio choosed weighted clusters which has some others advantages but cannot worked for your scenario.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.