Skip to content

Add support for request and response compression #765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add support for request and response compression #765

wants to merge 2 commits into from

Conversation

@Chaho12
Copy link
Member

@Chaho12 Chaho12 commented Sep 10, 2025 •
edited
Loading

Description

Accept-Encoding is removed when making POST request so response does not include compression.
Since client(trino-gateway) requests without It compression, trino server would return without compression (Content-Encoding).

It may depend on result, but when it seems compression is done about 51%(722->351), so approx 51% of network/memory could be saved.

Additional context and related issues

local test

before (add request/response header logging from main branch)

2025-09-10T11:28:12.860+0900    INFO    http-worker-80  io.trino.gateway.ha.handler.RoutingTargetHandler        Rerouting [http://[0:0:0:0:0:0:0:1]:8080/v1/statement]--> [http://localhost:8082/v1/statement]
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Request Headers:
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Accept] = [[*/*]]
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Content-Length] = [[480]]
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Content-Type] = [[application/json]]
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [User-Agent] = [[curl/8.7.1]]
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Via] = [[HTTP/1.1 TrinoGateway]]
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Forwarded-For] = [[[0:0:0:0:0:0:0:1]]]
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Forwarded-Host] = [[localhost]]
2025-09-10T11:28:12.861+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Forwarded-Port] = [[8080]]
2025-09-10T11:28:12.862+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Forwarded-Proto] = [[http]]
2025-09-10T11:28:12.862+0900    INFO    http-worker-80  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Trino-User] = [[lago-hive]]
2025-09-10T11:28:12.885+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Response Headers:
2025-09-10T11:28:12.886+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Date] = [[Wed, 10 Sep 2025 02:28:12 GMT]]
2025-09-10T11:28:12.886+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Vary] = [[Accept-Encoding]]
2025-09-10T11:28:12.886+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Content-Type] = [[application/json]]
2025-09-10T11:28:12.886+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [X-Content-Type-Options] = [[nosniff]]
2025-09-10T11:28:12.886+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Content-Length] = [[722]]

after

2025-09-10T11:20:41.950+0900    INFO    http-worker-81  io.trino.gateway.ha.handler.RoutingTargetHandler        Rerouting [http://[0:0:0:0:0:0:0:1]:8080/v1/statement]--> [http://localhost:8082/v1/statement]
2025-09-10T11:20:41.951+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Request Headers:
2025-09-10T11:20:41.953+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Accept] = [[*/*]]
2025-09-10T11:20:41.953+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Accept-Encoding] = [[gzip, deflate, br]]
2025-09-10T11:20:41.953+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Content-Length] = [[480]]
2025-09-10T11:20:41.953+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Content-Type] = [[application/json]]
2025-09-10T11:20:41.953+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [User-Agent] = [[curl/8.7.1]]
2025-09-10T11:20:41.957+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [Via] = [[HTTP/1.1 TrinoGateway]]
2025-09-10T11:20:41.957+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Forwarded-For] = [[[0:0:0:0:0:0:0:1]]]
2025-09-10T11:20:41.957+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Forwarded-Host] = [[localhost]]
2025-09-10T11:20:41.957+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Forwarded-Port] = [[8080]]
2025-09-10T11:20:41.957+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Forwarded-Proto] = [[http]]
2025-09-10T11:20:41.958+0900    INFO    http-worker-81  io.trino.gateway.proxyserver.ProxyRequestHandler        Header [X-Trino-User] = [[lago-hive]]
2025-09-10T11:20:41.997+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Response Headers:
2025-09-10T11:20:41.998+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Date] = [[Wed, 10 Sep 2025 02:20:41 GMT]]
2025-09-10T11:20:41.998+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Vary] = [[Accept-Encoding]]
2025-09-10T11:20:41.998+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Content-Type] = [[application/json]]
2025-09-10T11:20:41.998+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [X-Content-Type-Options] = [[nosniff]]
2025-09-10T11:20:41.998+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Content-Encoding] = [[gzip]]
2025-09-10T11:20:41.998+0900    INFO    http-client-proxy-52    io.trino.gateway.proxyserver.ProxyResponseHandler       Header [Content-Length] = [[351]]

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required, with the following suggested text:

* Fix some things.

@Chaho12 Chaho12 self-assigned this Sep 10, 2025
@cla-bot cla-bot bot added the cla-signed label Sep 10, 2025
@Chaho12 Chaho12 force-pushed the feature/jyoo/enable-compression branch 2 times, most recently from d4338a5 to 8e3975d Compare September 11, 2025 23:37
@Chaho12 Chaho12 marked this pull request as ready for review September 11, 2025 23:38
@Chaho12 Chaho12 changed the title Allow compression for response Add support for request and response compression Sep 11, 2025
ebyhr
ebyhr reviewed Sep 16, 2025
View reviewed changes
@Chaho12 Chaho12 force-pushed the feature/jyoo/enable-compression branch from 8e3975d to 3a7f15f Compare September 16, 2025 03:06
@Chaho12 Chaho12 requested a review from ebyhr September 16, 2025 08:11
ebyhr
ebyhr reviewed Sep 16, 2025
View reviewed changes
@Chaho12 Chaho12 force-pushed the feature/jyoo/enable-compression branch from 3a7f15f to 15c12e9 Compare September 16, 2025 23:44
@Chaho12 Chaho12 force-pushed the feature/jyoo/enable-compression branch from 15c12e9 to fb2dc46 Compare September 17, 2025 01:00
ebyhr
ebyhr reviewed Sep 18, 2025
View reviewed changes
@Chaho12 Chaho12 force-pushed the feature/jyoo/enable-compression branch from fb2dc46 to 0f0a82d Compare September 19, 2025 01:33
@Chaho12 Chaho12 requested a review from ebyhr September 19, 2025 01:35
@oneonestar
Copy link
Member

Looks like Airlift disabled Jetty's GZIPContentDecoder using a hack.
If we can make this optional in Airlift, we can avoid doing the heavy lifting.
Also, using Jetty's decoder is better since there are many possible encoding (zstd, compress...) and we only handle gzip here.

@oneonestar oneonestar mentioned this pull request Oct 2, 2025
Open
3 tasks
@Chaho12 Chaho12 force-pushed the feature/jyoo/enable-compression branch from 0f0a82d to 96d2ab9 Compare October 29, 2025 14:13
@Chaho12 Chaho12 requested a review from vishalya October 29, 2025 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@oneonestar oneonestar Awaiting requested review from oneonestar

@andythsu andythsu Awaiting requested review from andythsu

@ebyhr ebyhr Awaiting requested review from ebyhr

@vishalya vishalya Awaiting requested review from vishalya

Assignees

@Chaho12 Chaho12

Labels

cla-signed

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Chaho12 @oneonestar @ebyhr