atenet: cap Envoy worker concurrency - #1199
Open
eliranw wants to merge 2 commits into
Open
Conversation
With --concurrency unset Envoy sizes its worker pool from the host cpu count. Cgroup limits do not constrain that count, so on a many-core node Envoy opens more sockets than the default nofile allows and dies during startup with "Too many open files", leaving atenet-router and atenet-egress in a crashloop while the Go container beside each one stays healthy. Four workers suits both proxies today and, unlike the default, does not change with the machine underneath. Each proxy carries its own ENVOY_CONCURRENCY variable rather than sharing one value, because the two have different connection shapes. The router accepts short request/response connections. The egress gateway holds CONNECT tunnels open for the life of an actor's connection, and Envoy pins a connection to a worker for that lifetime, so a low count concentrates long tunnels on few workers. They start equal and can diverge later. Reading the count from a variable rather than a literal argument lets a deployment be resized without editing manifests. Kubernetes expands $(VAR) in command and args alike, so nothing wraps the entrypoint. The flag and its value are separate argv entries. Envoy's parser rejects --concurrency=N with "Couldn't find match for argument" and exits on a usage dump, so the joined spelling used elsewhere in these manifests for Go binaries does not carry over. atenet-egress-with-sdsmint.yaml carries a third Envoy, selected by --experimental-use-sdsmint rather than by an overlay, so it inherits nothing from the other two and takes the same treatment directly. Signed-off-by: Eliran Wolff <eliranw@nvidia.com>
Four worker threads suit the clusters we run today, but the right number follows the machine and the traffic, and changing it meant editing manifests. The router and the egress gateway take separate flags because their connection shapes differ: the router accepts short request/response connections, while the egress gateway holds CONNECT tunnels open for the life of an actor's connection. Both default to the manifest value. The override edits the live Deployments rather than the rendered YAML because the egress gateway is applied as a whole file rather than through an overlay, so there is no single render path to parameterize. Both Deployments run a Go container alongside Envoy, so the update is scoped with --containers=envoy rather than defaulting to every container. Under the agentgateway data plane there is no Envoy container and no variable to read, and the override skips. A value that differs from the manifest costs an extra rollout: the deploy applies the manifest, resetting the variable, and the override then sets it again. Requesting the manifest's own value, or passing no flag at all, patches nothing. This matches how --podcert-workers-per-signer behaves. Values are validated during argument parsing as well as at use, so a typo fails before any cluster work rather than after the install has run. Signed-off-by: Eliran Wolff <eliranw@nvidia.com>
eliranw
force-pushed
the
eliranw/atenet-envoy-concurrency-configurable
branch
from
August 25, 2026 18:41
a1de917 to
65f4e81
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1193
With
--concurrencyunset, Envoy sizes its worker pool from the host CPU count, which cgroup limits do not constrain. On a many-core node it exhausts the default nofile during startup and both atenet pods crashloop, with the Go container beside each one healthy:Each Envoy container now takes its worker count from a variable defaulting to 4, so a cluster can be resized without editing manifests:
hack/install-ate.shgains--atenet-router-concurrencyand--atenet-egress-concurrency, following the shape of--podcert-workers-per-signer. They are separate flags because Envoy pins a connection to a worker for its lifetime, and egress holds CONNECT tunnels open far longer than the router's request/response connections.atenet-egress-with-sdsmint.yamlcarries a third Envoy that no overlay reaches, so it is changed directly.No CPU requests or limits are added. #1193 lists deriving the count from a cpu request as the other option; the measurements against it are in a comment there.
Tested on kind: full install, both proxies healthy, and the flags verified against Envoy's own
server.concurrency. Whether 4 is the right number is still open, and #665 would answer it.Known gap
--atenet-router=agentgatewayruns no Envoy, so this does nothing there. agentgateway sizes its tokio runtime fromnum_cpusand needs its own change.