-
Notifications
You must be signed in to change notification settings - Fork 39
Description
Documentation for Custom S3 Endpoint Support
Related PR: benbjohnson/litestream#731
Once the PR is merged, we need to document the new custom S3 endpoint support feature that allows users to specify S3-compatible storage endpoints directly from the command line using URL query parameters.
Documentation Updates Needed
1. Update the Replication Guide
Add a new section about custom S3-compatible storage providers showing how to use query parameters.
URL Format
s3://bucket/path?endpoint=<endpoint>®ion=<region>&forcePathStyle=<true|false>&skipVerify=<true|false>
Query Parameters
| Parameter | Description | Default |
|---|---|---|
endpoint |
Custom S3-compatible endpoint URL | - |
region |
AWS region | us-east-1 for custom endpoints |
forcePathStyle |
Use path-style addressing | true for custom endpoints |
skipVerify |
Skip TLS certificate verification | false |
2. Add Provider Examples
MinIO
# Replicate to MinIO
litestream replicate /path/to/db.sqlite \
"s3://mybucket/db?endpoint=localhost:9000&forcePathStyle=true"
# Restore from MinIO
litestream restore -o restored.db \
"s3://mybucket/db?endpoint=localhost:9000"Tigris (Fly.io)
# Replicate to Tigris
litestream replicate /path/to/db.sqlite \
"s3://my-tigris-bucket/db?endpoint=fly.storage.tigris.dev®ion=auto"
# Restore from Tigris
litestream restore -o restored.db \
"s3://my-tigris-bucket/db?endpoint=fly.storage.tigris.dev®ion=auto"Wasabi
litestream replicate /path/to/db.sqlite \
"s3://mybucket/db?endpoint=s3.wasabisys.com®ion=us-east-1"Self-Hosted with Self-Signed Certificate
litestream replicate /path/to/db.sqlite \
"s3://mybucket/db?endpoint=storage.internal.company.com&skipVerify=true"3. Update the Restore Guide
Add examples showing restore operations with custom endpoints (as shown above).
4. Configuration File Alternative
Document that users can still use the configuration file for more complex setups:
dbs:
- path: /path/to/db.sqlite
replicas:
- type: s3
bucket: mybucket
path: db
endpoint: localhost:9000
region: us-east-1
force-path-style: true
skip-verify: true
access-key-id: YOUR_ACCESS_KEY
secret-access-key: YOUR_SECRET_KEY5. Backward Compatibility Note
Emphasize that all existing S3 URLs continue to work without modification:
# Standard AWS S3 - works exactly as before
litestream replicate /path/to/db.sqlite s3://mybucket/db
# Or with explicit region
litestream replicate /path/to/db.sqlite s3://mybucket.s3.us-west-2.amazonaws.com/db6. Supported Providers Section
Create a list of tested S3-compatible providers:
- MinIO
- Tigris (Fly.io)
- Wasabi
- Backblaze B2 (S3-compatible API)
- DigitalOcean Spaces
- Scaleway Object Storage
- Ceph with S3 gateway
- Any other S3-compatible storage service
Notes
- When using a custom endpoint,
forcePathStyledefaults totrueas most S3-compatible services require path-style addressing - The
regionparameter may be required by some services even if they don't use AWS regions - Use
skipVerify=trueonly for development or when you trust the self-signed certificate - If no scheme is provided in the endpoint,
http://is automatically prepended
This feature addresses the long-standing issue #219 and enables users to work with any S3-compatible storage provider directly from the command line without needing a configuration file.