Using cached EDS resources

[adisuissa]

Title: Using cached EDS resources

Description:
Currently after an EDS-cluster update, Envoy waits for an EDS response. If a timeout occurs, the EDS-cluster will be used with an empty assignment. This may break ongoing traffic, if the xDS server fails to reply in a timely manner. As noted in #13009 this can be solved if Envoy will cache the EDS resources.

Unfortunately Envoy cannot immediately use the cached resource because there are scenarios where the Cluster update requires an updated ClusterLoadAssignment (CLA), and using the cached resource will break traffic. An example is when a non-TLS cluster is upgraded to supporting TLS (see #11877, and this test) and using the previously assigned CLA will break traffic. A possible way to solve this while still adhering to the xDS protocol is to require that upon a material change to the Cluster the xDS-server will also modify the [Eds service_name]

envoy/api/envoy/config/cluster/v3/cluster.proto

Line 207 in 25c0e7c

string service_name = 2;

). However, many servers do not adhere to this, and to ensure backwards compatibility, a different approach is needed.

Suggested approach
The idea is to allow fallback to a cached EDS resource as follows:

1. When receiving a new/updated EDS-Cluster, wait for a ClusterLoadAssignment (CLA) for that cluster (Cluster is in a warming state).
2. If the CLA doesn't arrive within a certain timeframe, fetch the CLA resource from the cache, if exists, and make the Cluster active.
3. Otherwise, make the Cluster active with an empty assignment.

Note that this doesn't introduce a new way to break traffic, just prevents breaking traffic in the case where the CLA isn't modified and/or isn't sent from the xDS server. If a material change to the cluster occurs, but an updated assignment isn't received, traffic will be broken, just as happens today.

Plan

1. Introduce a cache for EDS resources.
2. Plumb the cache into the ClusterManagerFactory.
3. Update EdsClusterImpl to:
   • Store a resource in the cache.
   • Fetch the resource upon timeout.
   • Ensure that the resource TTL is enforced for cached resources.

Special considerations
The solution will need to clarify the lifetime of the cached resources.
The solution will target EDS resources only, as other resource (such as RDS) are currently "cached" and adhere to the protocol. In the future, the cache may be used for other types of resources, if needed.

Other implications
As with any cache, this will require additional memory. An additional O(N) memory will be required, where N is the number of CLA resources (or EDS clusters). Note that the resource is only accessed by the main thread (no copies to worker threads are needed).

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.