`.tgz` files are double-gzipped

[basil]

Jetty version(s)

12.0.14

Jetty Environment

EE 9
EE 10

Java version/vendor

openjdk version "17.0.12" 2024-07-16
OpenJDK Runtime Environment (build 17.0.12+7-Ubuntu-1ubuntu222.04)
OpenJDK 64-Bit Server VM (build 17.0.12+7-Ubuntu-1ubuntu222.04, mixed mode, sharing)

OS type/version

Ubuntu 22.04.5 LTS x86_64

Context

See JENKINS-73942.

Steps to reproduce

$ wget https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-home/12.0.14/jetty-home-12.0.14.tar.gz
$ tar -xzvf jetty-home-12.0.14.tar.gz
$ export JETTY_HOME=/path/to/jetty-home-12.0.14
$ git clone https://github.com/basil/jetty-tgz-repro.git
$ export JETTY_BASE=/path/to/jetty-tgz-repro

Observe the files are identical:

$ sha256sum $JETTY_BASE/webapps/static/resources/example.tar.gz $JETTY_BASE/webapps/static/resources/example.tgz
45749065add6664e1fc7c5a75665dd9dac85f70a1444612b895b222538601a13  webapps/static/resources/example.tar.gz
45749065add6664e1fc7c5a75665dd9dac85f70a1444612b895b222538601a13  webapps/static/resources/example.tgz

Start the server:

$ $JETTY_HOME/bin/jetty.sh start

Visit http://127.0.0.1:8080/static/ in Firefox and download both files.

Expected results

The files should be identical.

Actual results

The files are not identical, as the .tgz file is (erroneously!) double-gzipped. The .tar.gz file is correct.

$ sha256sum example.tar.gz example.tgz 
45749065add6664e1fc7c5a75665dd9dac85f70a1444612b895b222538601a13  example.tar.gz
1ca0c922947f4426b90526f3ec5427d2123b4b6fec92e66f8b14cec792271dae  example.tgz

Note

The problem only appears when the gzip module is enabled.

The problem disappeared after I unzipped $JETTY_HOME/lib/jetty-http-12.0.14.jar, updated org/eclipse/jetty/http/mime.properties to replace tgz=application/x-gtar with tgz=application/gzip, re-packed the JAR, and relaunched Jetty.

[janbartel]

@joakime is this as straightforward as it sounds?

[joakime]

@joakime is this as straightforward as it sounds?

Not sure.
The extension .tgz means "tar gzipped"

Compressing again is bad, sure, that part seems OK.
But the opposite direction, decompression, on Jetty Client, wouldn't be that easy.
I think we should also ignore the decompression side, as we don't want to get into the world of interpreting these shorthand multi-type extensions.

At first I was thinking this should be a fix in MimeTypes that the initialization of GzipHandler uses.
I think we need to treat this the same as *.svgz and hardcode it into the initialization of GzipHandler. (the hardcoded list is bothersome, wish we had a better way)

@basil in the meantime, as a proper workaround for Jenkins, start configuring your GzipHandler, and specify the exclude path of *.tgz and even the other compression types (*.jar, *.zip, etc ...)

[joakime]

The headers from the two downloads ...

First, the example.tgz

REQUEST

GET /static/example.tgz HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/png,image/svg+xml,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br, zstd
DNT: 1
Sec-GPC: 1
Connection: keep-alive
Referer: http://127.0.0.1:8080/static/
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
Priority: u=0, i

RESPONSE

HTTP/1.1 200 OK
Server: Jetty(12.0.14)
Vary: Accept-Encoding
Last-Modified: Tue, 22 Oct 2024 12:35:25 GMT
Content-Type: application/x-gtar
Accept-Ranges: bytes
Content-Encoding: gzip
Content-Length: 185

Next, the example.tar.gz

REQUEST

GET /static/example.tar.gz HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/png,image/svg+xml,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br, zstd
DNT: 1
Sec-GPC: 1
Connection: keep-alive
Referer: http://127.0.0.1:8080/static/
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
Priority: u=0, i

RESPONSE

HTTP/1.1 200 OK
Server: Jetty(12.0.14)
Last-Modified: Tue, 22 Oct 2024 12:35:25 GMT
Content-Type: application/gzip
Accept-Ranges: bytes
Content-Length: 170

[joakime]

Opened PR

• Issue #12397 - Add more default excludes to GzipHandler. #12414