Skip to content

feat: built-in reverse proxy for multi-instance local development#2026

Open
jlaneve wants to merge 8 commits intomainfrom
feat/portless-proxy
Open

feat: built-in reverse proxy for multi-instance local development#2026
jlaneve wants to merge 8 commits intomainfrom
feat/portless-proxy

Conversation

@jlaneve
Copy link
Contributor

@jlaneve jlaneve commented Mar 3, 2026

Summary

Adds a built-in reverse proxy (inspired by vercel-labs/portless) that routes <project>.localhost:6563 to the correct local Airflow instance. This eliminates port collisions when running multiple projects simultaneously — no more manually configuring ports in each project's config.

  • Always on by default — every instance gets a .localhost hostname routed through the proxy
  • Smart port defaults: backend port defaults to 8080 if it's free (existing scripts/muscle memory still work), otherwise a random port is allocated
  • --no-proxy escape hatch to disable and use classic fixed-port behavior
  • Auto-lifecycle: proxy starts on first astro dev start, stops when last project stops, auto-restarts on crash
  • Works with both Docker and standalone modes
  • Worktree-aware URLs: <worktree>.<repo>.localhost when inside a git worktree
  • Port 6563 = H-alpha emission line wavelength in angstroms

User experience

Before (today)

Running two projects simultaneously requires manual port configuration to avoid collisions:

# Project A — uses default ports
$ cd ~/projects/alpha && astro dev start
➤ Airflow UI: http://localhost:8080

# Project B — port 8080 is taken, need to manually configure
$ cd ~/projects/beta && astro config set webserver.port 8081
$ astro config set postgres.port 5433
$ astro dev start
➤ Airflow UI: http://localhost:8081

After (with this PR)

Just astro dev start in each project. The first project keeps port 8080 (the default), while subsequent projects get random ports — but you access everything via the proxy URL and don't need to care about backend ports:

# First project — port 8080 is free, keeps the default
$ cd ~/projects/alpha && astro dev start
➤ Airflow UI: http://alpha.localhost:6563

# Second project — port 8080 is taken, random port allocated automatically
$ cd ~/projects/beta && astro dev start
➤ Airflow UI: http://beta.localhost:6563

Both are accessible simultaneously in the browser at their .localhost URLs. No configuration needed. Single-instance users get the same default port (8080) they always had.

Git worktrees

When working in a git worktree, the hostname includes both the worktree name and the repo name for clarity:

$ cd ~/repos/astro-cli/.claude/worktrees/feature-branch
$ astro dev start
➤ Airflow UI: http://feature-branch.astro-cli.localhost:6563

Detection is pure filesystem — checks whether .git is a file (worktree) vs directory (normal repo). Works cross-platform; all modern browsers resolve *.localhost to 127.0.0.1 per RFC 6761.

Finding your ports

astro dev proxy status shows all running projects and their ports:

$ astro dev proxy status
✔ Proxy is running (PID 12345) on port 6563

Active routes:
URL                                          Backend Port    Postgres Port   Project Dir             PID
http://alpha.localhost:6563                  8080            5432            /Users/me/projects/alpha 0
http://beta.localhost:6563                   17291           18903           /Users/me/projects/beta  0

Opting out

If you prefer the old behavior with fixed ports and no proxy:

$ astro dev start --no-proxy

There is no config toggle — the proxy is always on unless --no-proxy is explicitly passed.

Crash recovery

If the proxy daemon crashes, the next astro dev proxy status auto-restarts it:

$ astro dev proxy status
Proxy is not running. Restarting…
✔ Proxy restarted (PID 56789) on port 6563

All routes are preserved (stored in ~/.astro/proxy/routes.json), so every project is immediately accessible again.

Architecture

Browser: http://my-project.localhost:6563
         ↓
    Proxy (port 6563) ── routes.json ── {hostname → port}
         ↓
    127.0.0.1:{8080 or random}  (webserver container)

Postgres: 127.0.0.1:{5432 or random}  (direct, shown in `proxy status`)

Key design decisions

Decision Choice Rationale
Proxy default Always on (--no-proxy to disable) Proxy is the primary UX; users access everything via .localhost URLs
Port allocation Try default first (8080/5432), random only on conflict Single-instance users keep familiar ports; multi-instance gets automatic allocation
Worktree detection Filesystem (.git file check) No dependency on git CLI or go-git; cross-platform
Docker route pruning Mode-aware (Mode: "docker" skips PID check) CLI exits after docker compose up, so PID-based pruning would incorrectly remove routes
Proxy state File-based (routes.json) Survives daemon crashes; enables auto-restart with no route loss
Route locking flock-based with atomic writes Correct concurrent access; no stale directory locks
Postgres routing Not proxied (shown in status) Postgres wire protocol isn't HTTP; TCP proxying adds complexity for marginal benefit

New files

  • airflow/proxy/proxy.go — HTTP reverse proxy server with landing page and 404
  • airflow/proxy/routes.go — Route registry with flock-based locking and atomic writes
  • airflow/proxy/daemon.go — Daemon lifecycle (start/stop/ensure/auto-stop)
  • airflow/proxy/ports.go — Smart port allocator: try defaults first, random fallback (10000-19999)
  • airflow/proxy/hostname.go — Hostname derivation with git worktree detection
  • cmd/airflow_proxy.goastro dev proxy status/stop/serve subcommands

Modified files

  • airflow/docker.go — Proxy integration in Start/Stop/Kill; smart port defaulting
  • airflow/standalone.go — Proxy integration in Start/Stop; smart port defaulting
  • airflow/container.goPortOverrides struct for port injection
  • cmd/airflow.go--no-proxy flag
  • config/config.go / config/types.goproxy.port config key

Test plan

  • Unit tests for route management, port allocation, hostname derivation (including worktree cases), proxy routing
  • IsPortAvailable exported wrapper tests
  • Single project: astro dev start → proxy auto-starts → browser opens http://{name}.localhost:6563 → backend uses port 8080
  • Multi-project: first project gets 8080, second gets random port; both accessible via .localhost hostnames
  • Git worktree: URL is <worktree>.<repo>.localhost
  • Stop/cleanup: last project stop triggers proxy shutdown
  • Opt-out: --no-proxy restores original fixed-port behavior
  • Crash recovery: astro dev proxy status auto-restarts dead daemon
  • Linux browser resolution of *.localhost

🤖 Generated with Claude Code

jlaneve and others added 5 commits March 3, 2026 14:06
Adds a reverse proxy that routes <project>.localhost:6563 to the correct
local Airflow instance, eliminating port collisions when running multiple
projects simultaneously. The proxy auto-starts on first `astro dev start`
and auto-stops when the last project stops.

Key features:
- HTTP reverse proxy routing by Host header (port 6563)
- Random port allocation (10000-19999) for backend services
- File-based route registry (~/.astro/proxy/routes.json) with locking
- Daemon lifecycle management (auto-start, auto-stop, crash recovery)
- Landing page at localhost:6563 showing active routes
- `astro dev proxy status/stop` subcommands
- `--no-proxy` flag and `proxy.enabled` config to opt out
- Works with both Docker and standalone modes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix all golangci-lint issues:
- Pass Route by pointer to AddRoute (hugeParam)
- Use named constants for file permissions (mnd, gosec)
- Fix gofumpt formatting (import order, struct alignment, const alignment)
- Fix "marshalling" → "marshaling" (misspell)
- Replace nil with http.NoBody in tests (httpNoBody)
- Rewrite if-else chain as switch (ifElseChain)
- Extract duplicate proxy registration in standalone.go (dupl)
- Remove unused return from setupTestDir (unparam)

Add worktree-aware hostname derivation:
- Detect git worktrees via .git file (pure filesystem, no CLI/library deps)
- Worktree URLs: <worktree>.<repo>.localhost (e.g. feature-branch.astro-cli.localhost)
- Normal repos: <dir>.localhost (unchanged)
- Works cross-platform (modern browsers resolve *.localhost per RFC 6761)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Make the reverse proxy always-on by default (--no-proxy as escape hatch)
and try default ports (8080/5432) first, only falling back to random
allocation when they're occupied. This preserves muscle memory for
single-instance users while still supporting multi-instance development.

- Remove proxy.enabled config gating; proxy is on unless --no-proxy
- Export IsPortAvailable from proxy package for port probing
- Docker mode: try default API/postgres ports before random allocation
- Standalone mode: try default webserver port before random allocation
- Switch routes.go to flock-based locking and atomic writes
- Simplify proxy handler variable naming

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coveralls-official
Copy link

coveralls-official bot commented Mar 3, 2026

Pull Request Test Coverage Report for Build f45b57a8-5f6e-4ff2-bb75-3d040315b3d6

Details

  • 457 of 896 (51.0%) changed or added relevant lines in 10 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+0.2%) to 35.914%

Changes Missing Coverage Covered Lines Changed/Added Lines %
airflow/container.go 8 18 44.44%
airflow/proxy/hostname.go 52 62 83.87%
airflow/proxy/ports.go 19 29 65.52%
airflow/standalone.go 50 73 68.49%
airflow/proxy/proxy.go 96 141 68.09%
airflow/proxy/routes.go 107 157 68.15%
cmd/airflow_proxy.go 41 97 42.27%
airflow/proxy/daemon.go 37 119 31.09%
airflow/docker.go 38 191 19.9%
Files with Coverage Reduction New Missed Lines %
airflow/docker.go 1 77.25%
Totals Coverage Status
Change from base Build f7e840d0-a911-4769-8b91-42a15a428d14: 0.2%
Covered Lines: 24288
Relevant Lines: 67629

💛 - Coveralls

@jlaneve jlaneve marked this pull request as ready for review March 3, 2026 23:05
@jlaneve jlaneve requested a review from a team as a code owner March 3, 2026 23:05
jlaneve and others added 3 commits March 3, 2026 21:00
- Add HTTP server timeouts (ReadHeaderTimeout, WriteTimeout, IdleTimeout)
  to prevent slowloris attacks and goroutine leaks
- Add graceful shutdown via SIGTERM/SIGINT signal handler with 5s grace
- Cache reverse proxy instances per backend port with shared transport
  instead of allocating a new one per request
- Store CLI version in PID file and restart daemon on version mismatch
  to prevent incompatibilities after CLI upgrades
- Log warnings in removeProxyRoute instead of silently swallowing errors
- Inline trivial proxyEnabled() wrapper
- Fix readRoutes to handle whitespace-only routes files gracefully

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant