Skip to content

Apache httpd module implementing RFC 9530 Digest Fields — adds Content-Digest and Repr-Digest headers from sidecar checksum files

License

Notifications You must be signed in to change notification settings

ptudor/mod-digest-fields

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

mod_digest_fields

An Apache module that adds RFC 9530 Content-Digest and Repr-Digest headers to HTTP responses, enabling clients to verify file integrity without a separate request.

When Apache serves a file, the module checks for a sidecar checksum file alongside it. If one exists, the hash is included in the response header using the standard Digest Fields format.

GET /releases/myapp-2.1.tar.gz HTTP/1.1

HTTP/1.1 200 OK
Content-Digest: sha-256=:VnYLtiDf1sVMSehoMeoYSw0u0cHNKh7A+4XSmaGSpEc=:
Content-Length: 10485760

The client now has a cryptographic hash of the file before it even finishes downloading.

Live Example

https://www.any53.com/ uses this module in production.

See It Work

Server side: publish a file with its checksum

# Create a file to serve
echo "Hello, RFC 9530!" > /var/www/files/greeting.txt

# Create the sidecar checksum (one command)
sha256sum /var/www/files/greeting.txt > /var/www/files/greeting.txt.sha256

# That's it. The sidecar file looks like this:
cat /var/www/files/greeting.txt.sha256
# 1bfa3474a00891becf8e2b8b3bba5da1e3ed25da51e6a09988e72cce74c1d758  /var/www/files/greeting.txt

Add two lines to your Apache config:

<Directory "/var/www/files">
    DigestFields On
</Directory>

Reload Apache, then request the file:

$ curl -si http://localhost/files/greeting.txt

HTTP/1.1 200 OK
Content-Digest: sha-256=:G/o0dKAIkb7PjiuLO7pdoePtJdpR5qCZiOcsznTB11g=:
Content-Length: 18
Content-Type: text/plain

Hello, RFC 9530!

The Content-Digest header is the SHA-256 hash of the file, base64-encoded per RFC 8941 Structured Fields.

Client side: verify the download

A verify.py script is included that downloads a file and checks it against the Content-Digest header in one step:

$ python3 verify.py http://localhost/files/greeting.txt
Fetching http://localhost/files/greeting.txt ...
Content-Digest: sha-256=:G/o0dKAIkb7PjiuLO7pdoePtJdpR5qCZiOcsznTB11g=:

  Algorithm : sha-256
  Expected  : 1bfa3474a00891becf8e2b8b3bba5da1e3ed25da51e6a09988e72cce74c1d758
  Got       : 1bfa3474a00891becf8e2b8b3bba5da1e3ed25da51e6a09988e72cce74c1d758

MATCH -- file integrity verified.

The script does exactly what a client should do:

  1. Download the file and read the Content-Digest header
  2. Parse the algorithm name and base64-encoded hash from the header
  3. Hash the downloaded bytes with the same algorithm
  4. Compare the two -- if they match, the file is intact

No dependencies beyond Python 3. Here's the full script (verify.py):

#!/usr/bin/env python3
"""Verify a download using RFC 9530 Content-Digest headers."""

import hashlib, base64, re, sys, urllib.request

# The algorithms we know how to verify
ALGORITHMS = {
    "sha-256": hashlib.sha256,
    "sha-512": hashlib.sha512,
}

def parse_content_digest(header):
    """
    Parse an RFC 9530 Content-Digest header value.
    Input:  'sha-256=:G/o0dKAIkb7PjiuLO7pdoePtJdpR5qCZiOcsznTB11g=:'
    Output: ('sha-256', b'\x1b\xfa\x34\x74...')  (algorithm name, raw hash bytes)
    """
    m = re.match(r'(sha-(?:256|512))=:([A-Za-z0-9+/=]+):', header)
    if not m:
        return None, None
    return m.group(1), base64.b64decode(m.group(2))

# Step 1: Download the file and grab the Content-Digest header
url = sys.argv[1]
print(f"Fetching {url} ...")

req = urllib.request.Request(url)
with urllib.request.urlopen(req) as resp:
    body = resp.read()
    header = resp.getheader("Content-Digest")

if not header:
    print("No Content-Digest header in response.")
    sys.exit(1)

print(f"Content-Digest: {header}")

# Step 2: Parse the header to get the algorithm and the expected hash
algo_name, expected = parse_content_digest(header)

# Step 3: Hash the bytes we actually received
actual = ALGORITHMS[algo_name](body).digest()

# Step 4: Compare
print(f"\n  Algorithm : {algo_name}")
print(f"  Expected  : {expected.hex()}")
print(f"  Got       : {actual.hex()}\n")

if actual == expected:
    print("MATCH -- file integrity verified.")
else:
    print("MISMATCH -- file may be corrupted or tampered with.")
    sys.exit(1)

Compressed files: both headers at once

For mirror servers hosting pre-compressed archives, the module can emit both headers -- one for the compressed file on disk, and one for the uncompressed original:

# On the server
sha256sum archive.tar.gz  > archive.tar.gz.sha256   # hash of compressed file
sha256sum archive.tar     > archive.tar.sha256       # hash of original
<Directory "/var/www/mirror">
    DigestFields On
    DigestFieldsRepr On
    SetEnv no-gzip 1
</Directory>
$ curl -si http://localhost/mirror/archive.tar.gz

HTTP/1.1 200 OK
Content-Digest: sha-256=:kF3mVaSgX1OwBxHj6qMKQRjOCPRtkHmP7mNJKAergNQ=:
Repr-Digest: sha-256=:G/o0dKAIkb7PjiuLO7pdoePtJdpR5qCZiOcsznTB11g=:
  • Content-Digest = hash of archive.tar.gz (the bytes on the wire)
  • Repr-Digest = hash of archive.tar (the uncompressed representation)

How It Works

The module uses sidecar files -- small text files containing a checksum, stored next to the content they describe:

/var/www/files/
    data.json               <- the file Apache serves
    data.json.sha256        <- the sidecar (contains the SHA-256 hash)

When a request arrives for data.json, the module:

  1. Checks if data.json.sha256 exists
  2. Reads the hex hash from it
  3. Converts hex to base64 per RFC 8941
  4. Adds the Content-Digest header to the response

If no sidecar exists, no header is added. No errors, no noise.

Sidecar file format

The module accepts common checksum formats. Any of these work:

# Plain hash
5676bbb620dfd6c54c49e86831ea2577aa8d9cbc7e6ad5ea1f6848e9bc4f69fa

# GNU coreutils (sha256sum output)
5676bbb620dfd6c54c49e86831ea2577aa8d9cbc7e6ad5ea1f6848e9bc4f69fa  filename.txt

# BSD (shasum -a 256 / openssl dgst)
SHA256 (filename.txt) = 5676bbb620dfd6c54c49e86831ea2577aa8d9cbc7e6ad5ea1f6848e9bc4f69fa

Validation is strict: the file must be a single line, under 1024 bytes, with exactly one hex string of the expected length. If anything is ambiguous, no header is emitted.

Build and Install

make
sudo make install

Or manually with apxs:

apxs -c mod_digest_fields.c
sudo apxs -i -a mod_digest_fields.so

Then load the module:

LoadModule digest_fields_module modules/mod_digest_fields.so

Configuration Reference

All directives work in server config, virtual host, <Directory>, and .htaccess (with FileInfo override).

DigestFields On|Off

Enable or disable the module. Default: Off.

<Directory "/var/www/downloads">
    DigestFields On
</Directory>

DigestFieldsAlgorithm

Hash algorithm to use. Checked in order; first sidecar found wins. Default: sha-256.

# Prefer SHA-512, fall back to SHA-256
DigestFieldsAlgorithm sha-512 sha-256
Algorithm Sidecar Extension Hex Length
sha-256 .sha256 64 chars
sha-512 .sha512 128 chars

DigestFieldsRepr On|Off

Also emit a Repr-Digest header for compressed files. The module strips the compression extension and looks for that file's sidecar. Default: Off.

DigestFields On
DigestFieldsRepr On

Supported compression extensions (default): .gz, .bz2, .xz, .zst, .lz4, .lzma, .lzfse, .br

DigestFieldsCompression

Override the default compression extension list. Use bare names without dots.

# Only recognize .gz and .zst as compression extensions
DigestFieldsCompression gz zst

DigestFieldsMatch

Only add headers for files matching a regex pattern (matched against the basename, not the full path).

# Only hash archives
DigestFieldsMatch "\.(tar\.gz|tar\.xz|tar\.zst|zip|iso)$"

DigestFieldsDirectory

Store sidecar files in a subdirectory instead of alongside the content. Keeps directory listings clean.

# Sidecar for /files/data.json is /files/.checksum/data.json.sha256
DigestFieldsDirectory .checksum

Client Algorithm Negotiation

Per RFC 9530, clients can request specific algorithms using the Want-Content-Digest and Want-Repr-Digest headers:

curl -H "Want-Content-Digest: sha-512=1, sha-256=0.5" http://localhost/files/data.json

The module respects client weights, filtered to algorithms configured on the server.

Example Configurations

Mirror server

Pre-compressed archives with integrity verification. Disable on-the-fly compression so the Content-Digest matches the bytes on the wire.

<Directory "/var/www/mirror">
    DigestFields On
    DigestFieldsRepr On
    DigestFieldsMatch "\.(tar\.gz|tar\.xz|tar\.zst|zip)$"
    SetEnv no-gzip 1
    SetEnv no-brotli 1
</Directory>

Clean directory listings

Hide sidecar files in a subdirectory so they don't clutter mod_autoindex output.

<Directory "/var/www/releases">
    DigestFields On
    DigestFieldsDirectory .checksum
</Directory>

Sidecar layout:

releases/
    myapp-2.1.tar.gz
    myapp-2.0.tar.gz
    .checksum/
        myapp-2.1.tar.gz.sha256
        myapp-2.0.tar.gz.sha256

SHA-512 with SHA-256 fallback

<Directory "/var/www/secure">
    DigestFields On
    DigestFieldsAlgorithm sha-512 sha-256
</Directory>

The module checks for .sha512 first. If not found, it tries .sha256.

Generating Sidecar Files

# Single file
sha256sum myapp-2.1.tar.gz > myapp-2.1.tar.gz.sha256

# All files in a directory
for f in /var/www/mirror/*.tar.gz; do
    sha256sum "$f" > "$f.sha256"
done

# Into a subdirectory
mkdir -p /var/www/mirror/.checksum
for f in /var/www/mirror/*.tar.gz; do
    sha256sum "$f" > "/var/www/mirror/.checksum/$(basename "$f").sha256"
done

# Both compressed and uncompressed (for Repr-Digest)
sha256sum archive.tar    > archive.tar.sha256
sha256sum archive.tar.gz > archive.tar.gz.sha256

On FreeBSD, use shasum -a 256 or sha256 instead of sha256sum. On macOS, use shasum -a 256. All output formats are supported.

Important Notes

mod_deflate / mod_brotli

If Apache compresses a response on the fly, the Content-Digest header will contain the hash of the original file, not the compressed bytes the client receives. This violates RFC 9530 semantics.

Solution: Disable on-the-fly compression for directories where DigestFields is enabled:

SetEnv no-gzip 1
SetEnv no-brotli 1

This is the expected configuration for mirror servers serving pre-compressed archives.

Symlinks

The module follows symlinks, consistent with Apache's FollowSymLinks directive. No special handling is performed.

Missing sidecars

If a sidecar file doesn't exist, no header is added. This is silent and intentional -- the absence of a Content-Digest header simply means "not verified." Clients can decide whether to proceed.

References

License

Apache License, Version 2.0. See LICENSE.

About

Apache httpd module implementing RFC 9530 Digest Fields — adds Content-Digest and Repr-Digest headers from sidecar checksum files

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published