Context
#508 moved every tenant tunable into the hot-reloadable settings directory, including the auth verifier wiring (auth.jwks_url, auth.role_claim) and the ClickHouse wiring. The three secrets stayed in boot config on purpose, because the settings directory is a tracked, hand-edited JSON directory and secrets don't belong in it:
auth.jwt_secret / WH_AUTH_JWT_SECRET
auth.operator_key / WH_AUTH_OPERATOR_KEY
clickhouse.password / WH_CH_PASSWORD
Today they are read once at boot from config.yaml or env and combined with the adopted wiring on every reconnect / verifier swap. Rotating any of them is a restart.
Raised by @EricAndrechek in review of #508: these are among the most frequently changed values for a new deployment trialing auth, and they need a story that isn't "plaintext in a tracked file".
Questions to settle
- Where do secrets live: env only (status quo),
*_file keys pointing at a mounted secret (the Docker/Kubernetes secret-mount convention), a separate secrets file outside the watched settings directory, or an external provider?
- Should rotation be hot: a re-read of the secret source on
SIGHUP / POST /v1/ops/settings/reload without a restart? The reload machinery (settings.Store.AfterAdopt, auth.Authenticator.Reconfigure, chconn.Manager.Reconfigure) already swaps the consumers, so the missing piece is only a secret source that can be re-read.
- Cloud: how the control plane delivers secrets to a node if it writes the settings directory via fan-out and secrets must not go through that path.
Out of scope
Token/key hardening in general is #228; this issue is only about where the three boot secrets are stored and how they rotate.
Context
#508 moved every tenant tunable into the hot-reloadable settings directory, including the auth verifier wiring (
auth.jwks_url,auth.role_claim) and the ClickHouse wiring. The three secrets stayed in boot config on purpose, because the settings directory is a tracked, hand-edited JSON directory and secrets don't belong in it:auth.jwt_secret/WH_AUTH_JWT_SECRETauth.operator_key/WH_AUTH_OPERATOR_KEYclickhouse.password/WH_CH_PASSWORDToday they are read once at boot from
config.yamlor env and combined with the adopted wiring on every reconnect / verifier swap. Rotating any of them is a restart.Raised by @EricAndrechek in review of #508: these are among the most frequently changed values for a new deployment trialing auth, and they need a story that isn't "plaintext in a tracked file".
Questions to settle
*_filekeys pointing at a mounted secret (the Docker/Kubernetes secret-mount convention), a separate secrets file outside the watched settings directory, or an external provider?SIGHUP/POST /v1/ops/settings/reloadwithout a restart? The reload machinery (settings.Store.AfterAdopt,auth.Authenticator.Reconfigure,chconn.Manager.Reconfigure) already swaps the consumers, so the missing piece is only a secret source that can be re-read.Out of scope
Token/key hardening in general is #228; this issue is only about where the three boot secrets are stored and how they rotate.