Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -197,8 +197,9 @@ CREDENTIAL_SOURCE_KEY_VERSION='1'
# ==========================================================================
# The front door is a magic link, so a deployment with no transport here
# has no way for anyone to log in. Set the SMTP block for a self-hosted
# install; RESEND_API_KEY is the hosted alternative and wins if both are
# set. None of this is required to get in — the first-run screen at /login
# install; Sendflare and Resend are HTTP alternatives. Sendflare wins if
# more than one provider is configured. None of this is required to get in —
# the first-run screen at /login
# creates an account with a password — and none of it has to be a file:
# the dashboard stores a relay too, and a stored one wins over everything
# here.
Expand Down Expand Up @@ -238,6 +239,12 @@ SMTP_FROM=''
# install.
RESEND_API_KEY=''

# worker · optional
# Never give this to: api, collector, query-gateway, realtime.
# Sendflare transactional email over HTTP. EMAIL_FROM must use a domain verified in
# Sendflare.
SENDFLARE_API_KEY=''

# ==========================================================================
# Configuring the deployment from the dashboard
# ==========================================================================
Expand Down
36 changes: 36 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ name: CI
# credentials. The token is pinned to read-only so a fork PR cannot use it.
permissions:
contents: read
packages: write

on:
push:
Expand All @@ -24,6 +25,41 @@ env:
NODE_VERSION: '24.18.0'

jobs:
# The 17old Zeabur deployment tracks this branch while provider support is
# being proven. Manual dispatch publishes only its worker image; normal CI on
# main and pull requests never enters this job.
worker-hotfix:
name: 17old Zeabur worker image
if: github.event_name == 'workflow_dispatch' && github.ref_name == 'agent/zeabur-selfhost-setup'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5

- uses: docker/setup-buildx-action@v3

- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: infra/docker/node-app.Dockerfile
push: true
build-args: |
APP=worker
GIT_COMMIT=${{ github.sha }}
tags: |
ghcr.io/17old-org/openanalytics/worker:${{ github.sha }}
ghcr.io/17old-org/openanalytics/worker:sendflare
cache-from: type=gha,scope=17old-worker
cache-to: type=gha,mode=max,scope=17old-worker
provenance: false

static:
name: Static checks
runs-on: ubuntu-latest
Expand Down
37 changes: 19 additions & 18 deletions SELF-HOSTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,8 @@ Two things worth knowing about the screen:
characters and nothing else, and leaving a password field empty keeps what is
stored.

The env blocks stay, and they stay first-class: `SMTP_HOST` alone activates SMTP
The env blocks stay, and they stay first-class: `SENDFLARE_API_KEY` activates
Sendflare's transactional HTTP API, while `SMTP_HOST` alone activates SMTP
with port 587, STARTTLS, no credential and `EMAIL_FROM` as the sender, and the
worker logs `email_transport_selected` at boot naming what it resolved and where
from. A deployment that would rather keep its configuration in files can set
Expand Down Expand Up @@ -558,23 +559,23 @@ Nothing here refuses to boot. Every one of these degrades a surface and says so
in the log, which is the deliberate shape: a deployment must not fail over a
feature it has not enabled.

| Missing | What breaks |
| ------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Mail transport (stored settings, `RESEND_API_KEY` or SMTP) on the worker | **Magic-link sign-in cannot complete**; invitations never arrive. Nothing is sent and the link is written nowhere. Sign in with the account the first-run screen made, then configure a relay in Account → Deployment. |
| `AUTH_TRUSTED_ORIGINS` on the api | **Every browser call from the dashboard is refused.** No `Access-Control-Allow-Origin` is emitted at all — fail-closed by design. |
| `APP_BASE_URL` on the api | Human-facing links (invitation acceptance, billing returns) point at pages the api does not serve. |
| `GOOGLE_*` / `GITHUB_*` | No Google/GitHub button. A provider appears only when both its id and secret are present. |
| `GEOIP_DB_PATH` | Every event carries null geo. No country, no city. |
| `CLICKHOUSE_MAINTENANCE_*` on the worker | **Site and account deletion queue and retry forever** instead of erasing anything. A wait, not a loss — but a silent one. |
| `STRIPE_SECRET_KEY` / `STRIPE_WEBHOOK_SECRET` | Billing surfaces disable themselves. Normally what a self-hosted install wants. |
| `OPENAI_API_KEY` **and** no stored provider | The AI assistant answers `503 not configured` — _before_ any question is charged — and the dashboard draws its chat button disabled. |
| `OA_CREDENTIAL_KEYRING` | Revenue-connection routes that encrypt are not mounted (404); reading and disconnecting still work. Account → Deployment closes with `no_keyring`, because it would be storing secrets it cannot protect. |
| `CREDENTIAL_SOURCE_SECRET` | No credential events are journalled at all. Reads are untouched. |
| `OBJECT_STORAGE_*` | Data import and export are not mounted. |
| `PREVIEW_TOKEN_*` | Rule preview is unavailable; the published rule set is served instead. A preview that cannot be authenticated is served as no preview, never as an unauthenticated one. |
| `REALTIME_CACHE_REDIS_URL` on the gateway | Replay defence becomes per-process — correct only with a single gateway instance. It warns. |
| `TELEGRAM_BOT_TOKEN` / `TELEGRAM_NOTIFY_CHAT_ID` | Notifications use the log transport. |
| `METRICS_REMOTE_WRITE_*` | No metrics pipeline; the structured-log metrics floor remains. |
| Missing | What breaks |
| --------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Mail transport (stored settings, `SENDFLARE_API_KEY`, `RESEND_API_KEY` or SMTP) on the worker | **Magic-link sign-in cannot complete**; invitations never arrive. Nothing is sent and the link is written nowhere. Sign in with the account the first-run screen made, then configure a relay in Account → Deployment. |
| `AUTH_TRUSTED_ORIGINS` on the api | **Every browser call from the dashboard is refused.** No `Access-Control-Allow-Origin` is emitted at all — fail-closed by design. |
| `APP_BASE_URL` on the api | Human-facing links (invitation acceptance, billing returns) point at pages the api does not serve. |
| `GOOGLE_*` / `GITHUB_*` | No Google/GitHub button. A provider appears only when both its id and secret are present. |
| `GEOIP_DB_PATH` | Every event carries null geo. No country, no city. |
| `CLICKHOUSE_MAINTENANCE_*` on the worker | **Site and account deletion queue and retry forever** instead of erasing anything. A wait, not a loss — but a silent one. |
| `STRIPE_SECRET_KEY` / `STRIPE_WEBHOOK_SECRET` | Billing surfaces disable themselves. Normally what a self-hosted install wants. |
| `OPENAI_API_KEY` **and** no stored provider | The AI assistant answers `503 not configured` — _before_ any question is charged — and the dashboard draws its chat button disabled. |
| `OA_CREDENTIAL_KEYRING` | Revenue-connection routes that encrypt are not mounted (404); reading and disconnecting still work. Account → Deployment closes with `no_keyring`, because it would be storing secrets it cannot protect. |
| `CREDENTIAL_SOURCE_SECRET` | No credential events are journalled at all. Reads are untouched. |
| `OBJECT_STORAGE_*` | Data import and export are not mounted. |
| `PREVIEW_TOKEN_*` | Rule preview is unavailable; the published rule set is served instead. A preview that cannot be authenticated is served as no preview, never as an unauthenticated one. |
| `REALTIME_CACHE_REDIS_URL` on the gateway | Replay defence becomes per-process — correct only with a single gateway instance. It warns. |
| `TELEGRAM_BOT_TOKEN` / `TELEGRAM_NOTIFY_CHAT_ID` | Notifications use the log transport. |
| `METRICS_REMOTE_WRITE_*` | No metrics pipeline; the structured-log metrics floor remains. |

---

Expand Down
16 changes: 11 additions & 5 deletions apps/worker/src/email-drain.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import {
/**
* Email delivery is a worker job (docs snapshot 02 §5): the API only writes the
* send to the outbox, and this loop drains it through whichever transport is
* configured — Resend, SMTP, or the log transport when neither is.
* configured — Sendflare, Resend, SMTP, or the log transport when none is.
*
* Since migration 0043 there are two places a transport can come from, and this
* file is where they are reconciled. **A transport stored in the database wins
Expand Down Expand Up @@ -143,10 +143,15 @@ export function startEmailDrain(deps: EmailDrainDeps): EmailDrain {
if (transport !== undefined && next === currentFingerprint) return transport

transport = selectEmailTransport({
// A stored relay wins over `RESEND_API_KEY` as well as over `SMTP_*`. The
// env-vs-env tie still goes to Resend (`selectEmailTransport`), so nothing
// about our own deployment changes: it stores nothing here.
...(stored ? {} : { apiKey: deps.env.RESEND_API_KEY }),
// A stored relay wins over provider keys as well as over `SMTP_*`. Among
// environment transports Sendflare wins, then Resend, then SMTP; every
// conflict is logged by `selectEmailTransport`.
...(stored
? {}
: {
sendflareApiKey: deps.env.SENDFLARE_API_KEY,
apiKey: deps.env.RESEND_API_KEY,
}),
smtp: block,
defaultFrom: deps.env.EMAIL_FROM ?? 'noreply@localhost',
log,
Expand All @@ -161,6 +166,7 @@ export function startEmailDrain(deps: EmailDrainDeps): EmailDrain {
// in the log finds nothing.
if (transport.id === 'log') {
const missing = [
...(deps.env.SENDFLARE_API_KEY ? [] : ['SENDFLARE_API_KEY']),
...(deps.env.RESEND_API_KEY ? [] : ['RESEND_API_KEY']),
...(deps.env.SMTP_HOST ? [] : ['SMTP_HOST']),
]
Expand Down
76 changes: 76 additions & 0 deletions docs/17old-zeabur-readiness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# 17old Zeabur readiness

This repository is the 17old-org fork of
[OpenLabs-so/openanalytics](https://github.com/OpenLabs-so/openanalytics),
tracked from upstream. The deployment baseline is the upstream release tag
`v0.4.2`; do not deploy a floating `main` commit as though it were a release.

## Decision recorded before a server is purchased

OpenAnalytics is a multi-service application, not a single Node process. Its
supported self-hosted layout contains Postgres, ClickHouse, two Valkey
instances, migrations, tracker build, API, collector, worker, query gateway,
realtime, web, and a proxy. The stock deployment is Docker Compose and needs
four public hostnames:

- `app.<base-domain>` -- dashboard
- `api.<base-domain>` -- API and OAuth callbacks
- `c.<base-domain>` -- collector and `oa.js`
- `rt.<base-domain>` -- realtime SSE

Zeabur does **not** directly deploy Docker Compose YAML. Therefore do not
create one generic "Node.js" service from this repository. At deployment time
we will choose one of these explicit paths:

1. Convert the supported topology to a Zeabur Template and deploy it as
separate services with persistent volumes. This keeps Zeabur's managed
server, networking, logs, and service controls.
2. Run the upstream Docker Compose stack on a standalone Docker host. This is
the upstream-supported route, but it is not a Zeabur-managed service
deployment.

The first path is the intended one for a newly purchased Zeabur Server. No
secrets, domains, or service definitions have been generated yet, because they
must be bound to the final domain and deployment topology.

## Minimum server to buy

Buy an **amd64/x86_64** server with **2 vCPU, 8 GB RAM, 80 GB SSD/NVMe, and one
public IPv4 address**. This is the smallest configuration I recommend for the
Zeabur-managed path:

- Upstream documents about 4 GB RAM for the OpenAnalytics runtime itself.
- A Zeabur Server also runs K3s and needs headroom; 4 GB total would leave no
dependable operating margin for the database containers and the platform.
- Official release images are amd64-only. Choosing ARM would force a local
build of all images and materially raises the memory requirement.
- 80 GB is a starting disk allocation, not an analytics-data limit. ClickHouse
data and backups grow with traffic, so plan to expand storage or retain fewer
snapshots when disk usage reaches 70%.

For a throwaway, directly managed Docker evaluation (not the Zeabur-managed
path), 2 vCPU / 4 GB RAM / 50 GB SSD can run the prebuilt release images. Do
not use that smaller profile for the long-lived Zeabur deployment.

## Purchase and DNS checklist

Before deployment, provide these non-secret inputs:

1. The chosen base domain, for example `analytics.17old.org`.
2. Control of DNS so the four names above can point at the new server.
3. A Zeabur workspace/project where the server can be selected.
4. Confirmation that the server has the required public networking and that
Zeabur can reserve resources for stateful services.

Do not put generated `infra/selfhost/.env`, `infra/selfhost/env/*.env`, or
`infra/selfhost/docker-compose.override.yml` in Git. They contain database
passwords and signing keys. The upstream generator is intentionally run only
once against the final domain and refuses to overwrite an existing install.

## Source and license obligations

The code is AGPL-3.0-only. If this fork is modified and offered as a network
service, users must be offered the corresponding source. Keep this repository
public or provide an equivalent source offer in the running dashboard. Also
use a 17old-owned domain and visual identity: the upstream OpenAnalytics name
and hosted-service branding are not granted by the code licence.
Loading
Loading