Expected behavior:

When using http2, the app access service should set the end_stream flag on the frame with the headers.

Current behavior:

When using http2, app access does not set the end_stream on a simple GET request that has only headers. This makes some upstream components error out in unexpected ways. Without the end_stream flag being set, the http2 server expects more data to come in a subsequent frame.

In particular, this causes difficulties when app access sends the request to another reverse proxy such as an ALB or an envoy endpoint. Both of those will treat the request like there is more data to come, and set the Transfer-Encoding: chunked header when it sends the request further upstream. Since the Transfer-Encoding: chunked header is not allowed on a simple GET request since it has no body, this request may well be rejected altogether.

Bug Details

Teleport version

10.2.2 and up

This is a regression. I have tested the following versions:

• 10.2.0 (works correctly)
• 10.2.1 (works correctly)
• 10.2.2 (triggers the problem)
• 10.2.4 (triggers the problem)
• 11.x.y (all affected)
• 12.x.y (all work correctly)

The version of the agent seems to be the deciding factor.

Recreation steps

Set up an http2 capable endpoint. envoy is good since it has the ability to really turn up debugging

Configure a teleport app access app to connect to the envoy https endpoint

Send a request over app access

Since envoy has good debugging, its logs will indicate that the end_stream is missing, and any upstream app from there will have the Transfer-Encoding: chunked header present.

I've included an example envoy.yaml (generate a self-signed cert), a teleport.yaml (app access agent), and the logs from envoy when sending the request through my lap setup.

envoy.yaml - ran as envoy -c /etc/envoy/envoy.yaml --log-level trace
envoylogs.txt
teleport.yaml

In particular, the [2023-03-28 21:55:23.742][19][debug][http] [source/common/http/conn_manager_impl.cc:972] [C38][S16903837606675993007] request headers complete (end_stream=false): log line indicates a problem. When using another http2 client to send traffic to envoy, this log indicates end_stream=true