The HTTP Garden

The HTTP Garden is a collection of HTTP servers and proxies configured to be composable, along with scripts to interact with them in a way that makes finding vulnerabilities much much easier. For some cool demos of the vulnerabilities that you can find with the HTTP Garden, check out our ShmooCon 2024 talk.

Acknowledgements

We'd like to thank our friends at Galois, Trail of Bits, Narf Industries, and Dartmouth College for making this project possible.

This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under contract number HR0011-19-C-0076.

Getting Started

Dependencies

0. The HTTP Garden runs on x86_64 Linux, and is untested on other platforms.
1. The target servers are built and run in Docker containers, so you'll need Docker.
2. You'll also need the following Python packages, which you can get from PyPI (i.e. with pip) or from your system package manager:

• docker
  • For interacting with Docker
• pyyaml
  • For parsing yaml
• tqdm
  • For progress bars

If you're installing Python packages with your system package manager, be aware that the package names may need to be prefixed with py3-, python3-, or python-, depending on the system.

3. I also highly recommend installing rlwrap from your package manager, because it makes the Garden repl a whole lot more fun.

Building

• Build the base image:

docker build ./images/http-garden-soil -t http-garden-soil

This image contains some basic utilities, plus a forked AFL++ that facilitates collecting coverage from processes without killing them.

• Build some HTTP servers and proxies:

docker compose build gunicorn hyper nginx haproxy nginx_proxy

There are, of course, way more targets in the HTTP garden than the ones we just built; it's just that building them all takes a long time. Even building these few will take a few minutes!

Running

• Start up some servers and proxies:

docker compose up gunicorn hyper nginx haproxy nginx_proxy

• Start the repl:

rlwrap python3 tools/repl.py

• Filter a basic GET request through HAProxy, then through an Nginx reverse proxy, then send the result to Gunicorn, Hyper, and Nginx origin servers, and display whether their interpretations match:

garden> payload 'GET / HTTP/1.1\r\nHost: whatever\r\n\r\n' # Set the payload
garden> transduce haproxy nginx_proxy # Run the payload through the reverse proxies
[1]: 'GET / HTTP/1.1\r\nHost: whatever\r\n\r\n'
    ⬇️ haproxy
[2]: 'GET / HTTP/1.1\r\nhost: whatever\r\n\r\n'
    ⬇️ nginx_proxy
[3]: 'GET / HTTP/1.1\r\nHost: echo\r\nConnection: close\r\n\r\n'
garden> servers gunicorn hyper nginx # Select the servers
garden> grid # Show their interpretations
         g
         u
         n
         i h n
         c y g
         o p i
         r e n
         n r x
        +-----
gunicorn|✓ ✓ ✓
hyper   |  ✓ ✓
nginx   |    ✓

Seems like they all agree. Let's try a more interesting payload:

garden> payload 'POST / HTTP/1.1\r\nHost: a\r\nTransfer-Encoding: chunked\r\n\r\n0\n\r\n'
garden> grid
         g
         u
         n
         i h n
         c y g
         o p i
         r e n
         n r x
        +-----
gunicorn|✓ ✓ X
hyper   |  ✓ X
nginx   |    ✓

There's a discrepancy! This is because Nginx supports \n as a line ending in chunk lines, but Hyper and Gunicorn don't. Nginx is technically violating RFC 9112 here, but the impact is likely minimal.

Directory Layout

images

The images directory contains a subdirectory for each HTTP server and transducer in the Garden. Each target gets its own Docker image. All programs are built from source when possible. So that we can easily build multiple versions of each target, all targets are paremetrized with a repository URL (APP_REPO), branch name (APP_BRANCH), and commit hash (APP_VERSION).

tools

The tools directory contains the scripts that are used to interact with the servers. Inside it, you'll find

• diagnose_anomalies.py: A script for enumerating benign HTTP parsing quirks in the systems under test to be ignored during fuzzing,
• repl.py: The primary user interface to the HTTP Garden,
• update.py: A script that updates the commit hashes in docker-compose.yml,
• ...and a few more scripts that aren't user-facing.

Targets

HTTP Servers

Name	Runs locally?	Coverage Collected?
aiohttp	yes	yes
apache_httpd	yes	yes
apache_tomcat	yes	no
cheroot	yes	yes
cpp_httplib	yes	no
cpython_stdlib	yes	no
dart_stdlib	yes	no
eclipse_grizzly	yes	no
eclipse_jetty	yes	no
fasthttp	yes	no
go_stdlib	yes	no
gunicorn	yes	yes
h2o	yes	yes
haproxy_fcgi	yes	no
hyper	yes	no
hypercorn	yes	yes
ktor	yes	no
libevent	yes	no
libmicrohttpd	yes	no
libsoup	yes	no
lighttpd	yes	yes
mongoose	yes	yes
netty	yes	no
nginx	yes	yes
node_stdlib	yes	no
openbsd_httpd	yes	no
openlitespeed	yes	no
openwrt_uhttpd	yes	yes
php_stdlib	yes	no
phusion_passenger	yes	no
protocol_http1	yes	no
puma	yes	no
servicetalk	yes	no
tornado	yes	no
twisted	yes	no
undertow	yes	no
uvicorn	yes	yes
waitress	yes	yes
webrick	yes	no
yahns	yes	no
iis	no	no

HTTP Transducers

Name	Runs locally?
apache_httpd_proxy	yes
apache_traffic_server	yes
busybox_httpd_proxy	yes
go_stdlib_proxy	yes
h2o_proxy	yes
haproxy	yes
haproxy_invalid	yes
lighttpd_proxy	yes
nginx_proxy	yes
openlitespeed_proxy	yes
pingora	yes
pound	yes
spring_cloud_gateway	yes
squid	yes
varnish	yes
yahns_proxy	yes
akamai	no
awselb_classic	no
awselb_application	no
cloudflare	no
google_classic	no
iis_proxy	no
openbsd_relayd	no

Omissions

The following are explanations for a few notable omissions from the Garden:

| Name | Rationale | | unicorn | Uses the same HTTP parser as yahns. | | SwiftNIO | Uses llhttp for HTTP parsing, which is already covered by node_stdlib. | | Bun | Uses picohttpparser for HTTP parsing, which is already covered by h2o. | | Deno | Uses hyper for HTTP parsing, which is already in the Garden. | | Daphne | Uses twisted for HTTP parsing, which is already in the Garden. | | pitchfork | Uses the same parser as yahns. | | nghttpx | Uses lhttp for HTTP parsing, which is already covered by node_stdlib. | | CherryPy | Uses cheroot for HTTP parsing, which is already in the Garden. | | libhttpserver | Uses libmicrohttpd for HTTP parsing, which is already in the Garden. | | Werkzeug | Uses the CPython stdlib for HTTP parsing, which is already in the Garden. | Caddy | Uses the Go stdlib for HTTP parsing, which is already in the Garden. | | Tengine | Uses Nginx's HTTP parsing logic. | | OpenResty | Uses Nginx's HTTP parsing logic. | | Google Cloud Global External Application Load Balancer | Based on Envoy. | | Google Cloud Regional External Application Load Balancer | Based on Envoy. |

Results

See TROPHIES.md for a complete list of bugs that the Garden has found.