feat(broker): store provider client secrets in GCP Secret Manager

.env.example

@@ -24,6 +24,15 @@ ENCRYPTION_KEY=
 STATE_KEY=
 
 API_KEY=nexus-admin-key
+# --- Provider client-secret storage (optional) ---
+# internal (default) keeps OAuth2 provider client secrets in Postgres.
+# gcp-secret-manager stores one Secret Manager secret per provider instead;
+# see docs/guides/provider-secret-backends.md and deploy/terraform/gcp/secret-backend.
+SECRET_BACKEND=internal
+# Required for gcp-secret-manager outside Google Cloud (on Cloud Run/GCE/GKE the metadata server supplies it).
+GCP_PROJECT_ID=
+GCP_SECRET_PREFIX=nexus-providers
+
 # Optional: mounted secret files for API key rotation without broker restart.
 # API_KEY_FILE may contain one key; API_KEYS_FILE may contain comma/newline-separated keys.
 API_KEY_FILE=

.github/workflows/ci.yml

@@ -33,7 +33,7 @@ jobs:
       - name: Setup Go
         uses: actions/setup-go@v5
         with:
-          go-version-file: nexus-gateway/go.mod
+          go-version-file: nexus-broker/go.mod   # the highest go directive in the repo; the others build with it
 
       - name: Setup Node.js
         uses: actions/setup-node@v4

.gitignore

@@ -60,3 +60,8 @@ site/
 docs/site/
 node_modules/
 __pycache__/
+
+# Terraform working directories (deploy/terraform/**)
+.terraform/
+*.tfstate
+*.tfstate.*

VERSION

@@ -1 +1 @@
-0.4.3
+0.5.0

deploy/terraform/gcp/secret-backend/README.md

@@ -0,0 +1,21 @@
+# GCP Secret Manager backend for nexus-broker
+
+Terraform for `SECRET_BACKEND=gcp-secret-manager`. It grants the broker's service account the least it needs to keep provider client secrets in Secret Manager: a custom role with only `secretmanager.secrets.create`, and a second custom role with only get, delete and version add/access/list/destroy, restricted by an IAM condition to secrets named `<prefix>-*`. Neither carries IAM-policy permissions.
+
+```hcl
+module "nexus_secret_backend" {
+  source                 = "github.com/Prescott-Data/nexus-framework//deploy/terraform/gcp/secret-backend?ref=main"
+  project_id             = var.project_id
+  broker_service_account = google_service_account.broker.email
+  secret_prefix          = "nexus-providers"
+}
+
+# Then on the broker:
+#   SECRET_BACKEND=gcp-secret-manager
+#   GCP_PROJECT_ID=<project_id>          (optional on Cloud Run, GCE and GKE)
+#   GCP_SECRET_PREFIX=nexus-providers
+```
+
+The broker checks the backend at startup and refuses to boot if it cannot reach Secret Manager under the prefix, so a missing binding shows up in the first log line rather than at the first OAuth exchange.
+
+See `docs/guides/provider-secret-backends.md` for how the backend behaves.

deploy/terraform/gcp/secret-backend/main.tf

@@ -0,0 +1,91 @@
+# Grants a nexus-broker service account exactly what SECRET_BACKEND=gcp-secret-manager
+# needs, and nothing it does not:
+#
+#   - secretmanager.secrets.create on the project. Create is authorised against
+#     the project, not the secret, so it cannot be scoped by name. The custom
+#     role below carries only that one permission; it cannot read anything.
+#   - A second custom role with only get, delete, and version add/access/
+#     list/destroy, bound with a condition on secrets whose name starts with
+#     the prefix. No IAM-policy or metadata-update permissions. Secrets
+#     outside the prefix (your ENCRYPTION_KEY, database URL, and so on) stay
+#     out of reach.
+#
+# Usage from your own Terraform:
+#
+#   module "nexus_secret_backend" {
+#     source                 = "github.com/Prescott-Data/nexus-framework//deploy/terraform/gcp/secret-backend?ref=main"
+#     project_id             = "my-project"
+#     broker_service_account = google_service_account.broker.email
+#     secret_prefix          = "nexus-providers"   # must equal GCP_SECRET_PREFIX
+#   }
+
+terraform {
+  required_version = ">= 1.5"
+  required_providers {
+    google = {
+      source  = "hashicorp/google"
+      version = ">= 5.0"
+    }
+  }
+}
+
+data "google_project" "this" {
+  project_id = var.project_id
+}
+
+resource "google_project_service" "secretmanager" {
+  count = var.enable_api ? 1 : 0
+
+  project            = var.project_id
+  service            = "secretmanager.googleapis.com"
+  disable_on_destroy = false
+}
+
+resource "google_project_iam_custom_role" "secret_creator" {
+  project     = var.project_id
+  role_id     = var.creator_role_id
+  title       = "Nexus provider secret creator"
+  description = "Lets nexus-broker create Secret Manager secrets. Read and write on existing secrets are granted separately, scoped by name prefix."
+  permissions = ["secretmanager.secrets.create"]
+}
+
+resource "google_project_iam_member" "secret_creator" {
+  project = var.project_id
+  role    = google_project_iam_custom_role.secret_creator.id
+  member  = "serviceAccount:${var.broker_service_account}"
+
+  depends_on = [google_project_service.secretmanager]
+}
+
+# Only what the broker does to a secret it owns: read metadata and delete the
+# secret, add/access/list/destroy versions. Unlike roles/secretmanager.admin
+# this carries no IAM-policy or metadata-update permissions, so a compromised
+# broker cannot grant access to its secrets or anyone else's.
+resource "google_project_iam_custom_role" "prefixed_secret_operator" {
+  project     = var.project_id
+  role_id     = var.operator_role_id
+  title       = "Nexus provider secret operator"
+  description = "Lets nexus-broker read, version and delete the provider secrets it created. Bound with a condition on the secret-name prefix."
+  permissions = [
+    "secretmanager.secrets.get",
+    "secretmanager.secrets.delete",
+    "secretmanager.versions.add",
+    "secretmanager.versions.access",
+    "secretmanager.versions.list",
+    "secretmanager.versions.destroy",
+  ]
+}
+
+resource "google_project_iam_member" "prefixed_secret_operator" {
+  project = var.project_id
+  role    = google_project_iam_custom_role.prefixed_secret_operator.id
+  member  = "serviceAccount:${var.broker_service_account}"
+
+  condition {
+    title       = "nexus-broker provider secrets only"
+    description = "Secrets named ${var.secret_prefix}-<provider uuid>, created by nexus-broker."
+    expression  = "resource.name.startsWith(\"projects/${data.google_project.this.number}/secrets/${var.secret_prefix}-\")"
+  }
+
+  depends_on = [google_project_service.secretmanager]
+}

deploy/terraform/gcp/secret-backend/outputs.tf

@@ -0,0 +1,13 @@
+output "secret_prefix" {
+  description = "Value to set as GCP_SECRET_PREFIX on the broker."
+  value       = var.secret_prefix
+}
+
+output "broker_env" {
+  description = "Environment variables that turn the backend on for the broker."
+  value = {
+    SECRET_BACKEND    = "gcp-secret-manager"
+    GCP_PROJECT_ID    = var.project_id
+    GCP_SECRET_PREFIX = var.secret_prefix
+  }
+}

deploy/terraform/gcp/secret-backend/variables.tf

@@ -0,0 +1,44 @@
+variable "project_id" {
+  description = "Project that hosts the broker's provider secrets. Set GCP_PROJECT_ID on the broker to the same value, or leave it unset on Cloud Run, GCE and GKE where the metadata server supplies it."
+  type        = string
+}
+
+variable "broker_service_account" {
+  description = "Email of the service account the nexus-broker runs as."
+  type        = string
+}
+
+variable "secret_prefix" {
+  description = "Prefix for secret IDs. Must equal the broker's GCP_SECRET_PREFIX. Letters, digits, '-' and '_' only."
+  type        = string
+  default     = "nexus-providers"
+
+  validation {
+    condition     = can(regex("^[A-Za-z0-9_-]+$", var.secret_prefix))
+    error_message = "secret_prefix may only contain letters, digits, '-' and '_'."
+  }
+
+  validation {
+    # Secret Manager IDs are limited to 255 characters; the broker appends "-" plus a 36-character UUID.
+    condition     = length(var.secret_prefix) <= 218
+    error_message = "secret_prefix may be at most 218 characters so that <prefix>-<provider uuid> fits Secret Manager's 255-character ID limit."
+  }
+}
+
+variable "enable_api" {
+  description = "Enable secretmanager.googleapis.com on the project. Set false if another module already manages project services."
+  type        = bool
+  default     = true
+}
+
+variable "creator_role_id" {
+  description = "ID for the custom role that carries secretmanager.secrets.create."
+  type        = string
+  default     = "nexusProviderSecretCreator"
+}
+
+variable "operator_role_id" {
+  description = "ID for the custom role that carries the per-secret permissions, bound with the prefix condition."
+  type        = string
+  default     = "nexusProviderSecretOperator"
+}

docker-compose.yml

@@ -67,6 +67,16 @@ services:
       REQUIRE_API_KEY: ${REQUIRE_API_KEY}
       REQUIRE_ALLOWLIST: ${REQUIRE_ALLOWLIST}
 
+      # Provider client-secret storage (see docs/guides/provider-secret-backends.md)
+      SECRET_BACKEND: ${SECRET_BACKEND:-internal}
+      GCP_PROJECT_ID: ${GCP_PROJECT_ID:-}
+      GCP_SECRET_PREFIX: ${GCP_SECRET_PREFIX:-nexus-providers}
+      # For SECRET_BACKEND=gcp-secret-manager on a workstation, mount your
+      # Application Default Credentials and point the client at them:
+      # GOOGLE_APPLICATION_CREDENTIALS: /gcp/adc.json
+    # volumes:
+    #   - ~/.config/gcloud/application_default_credentials.json:/gcp/adc.json:ro
+
   gateway:
     build:
       context: ./nexus-gateway

docs/CHANGELOG.md

@@ -12,6 +12,26 @@ All notable changes to Nexus are documented here. This project follows [Semantic
 
 <div class="changelog-release" markdown>
 
+## 0.5.0 <span class="changelog-date">2026-10-10</span>
+
+<div class="changelog-meta" markdown>
+<a class="changelog-release-link" href="https://github.com/Prescott-Data/nexus-framework/releases/tag/v0.5.0" target="_blank" rel="noopener noreferrer">View release on GitHub →</a>
+</div>
+
+**Added**
+
+- **Provider client secrets can live in GCP Secret Manager.** `SECRET_BACKEND=gcp-secret-manager` on the Broker stores each OAuth2 provider's `client_secret` as its own Secret Manager secret, named `<GCP_SECRET_PREFIX>-<provider id>`, and leaves `provider_profiles.client_secret` NULL. Registration rolls back if the backend write fails, `PATCH` rotates the secret and destroys earlier versions, and deleting a provider deletes its secret. The default `internal` backend is unchanged, and rows that still carry a column value keep working after the switch. The Broker checks the backend at startup and refuses to boot if it cannot reach it. A Terraform module in `deploy/terraform/gcp/secret-backend` grants the Broker's service account create on the project plus admin on prefixed secrets only. First backend for #63; the `pkg/vault/providers.Backend` interface is where Vault and AWS go next.
+
+**Changed**
+
+- The token encryption helpers moved from `pkg/vault` to `pkg/vault/tokens`, so that `pkg/vault` can hold both the token vault and the provider secret backends. `vault.Encrypt` and `vault.Decrypt` remain as deprecated wrappers. No behaviour change.
+
+</div>
+
+---
+
+<div class="changelog-release" markdown>
+
 ## 0.4.3 <span class="changelog-date">2026-10-10</span>
 
 <div class="changelog-meta" markdown>

docs/getting-started/configuration.md

@@ -38,6 +38,9 @@ Both the Broker and the Gateway must receive the same value for `STATE_KEY`. If
 | `REQUIRE_API_KEY` | No | When `true`, the Broker rejects requests without a valid `X-API-Key` header. Default: `true` |
 | `REQUIRE_ALLOWLIST` | No | When `true`, the Broker enforces `ALLOWED_CIDRS` for all requests. Default: `false` |
 | `PORT` | No | Port the Broker listens on. Default: `8080` |
+| `SECRET_BACKEND` | No | Where OAuth2 provider client secrets are stored. `internal` (default) keeps them in Postgres; `gcp-secret-manager` stores them in Google Cloud Secret Manager. See [Provider Secret Backends](../guides/provider-secret-backends.md). |
+| `GCP_PROJECT_ID` | Conditional | Project for `gcp-secret-manager`. Optional on Cloud Run, GCE and GKE. |
+| `GCP_SECRET_PREFIX` | No | Secret ID prefix for `gcp-secret-manager`. Default: `nexus-providers` |
 
 ---
 

docs/guides/provider-secret-backends.md

@@ -0,0 +1,69 @@
+---
+icon: material/safe
+---
+
+# Provider Secret Backends
+
+The Broker stores two kinds of secret, and they live in different places.
+
+| Secret | Where it lives | Protected by |
+|---|---|---|
+| Tokens and captured credentials for a connection | `tokens.encrypted_data` in Postgres | AES-256-GCM under `ENCRYPTION_KEY`; the database only ever sees ciphertext |
+| An OAuth2 provider's `client_secret` | `provider_profiles.client_secret` in Postgres by default, or an external **secret backend** | The backend's own access control |
+
+By default the provider's client secret sits in the same row as its metadata. That is fine for a single-team deployment, but many environments require application credentials to live only in an approved secret manager, and nobody wants client secrets turning up in a database dump. `SECRET_BACKEND` moves them out.
+
+## Configuration
+
+| Variable | Required | Description |
+|---|---|---|
+| `SECRET_BACKEND` | No | `internal` (default) keeps client secrets in Postgres. `gcp-secret-manager` stores them in Google Cloud Secret Manager. |
+| `GCP_PROJECT_ID` | Conditional | Project that holds the secrets. Optional on Cloud Run, GCE and GKE, where the metadata server supplies it. Required elsewhere. |
+| `GCP_SECRET_PREFIX` | No | Prefix for secret IDs. Default `nexus-providers`. Letters, digits, `-` and `_` only. |
+
+The Broker authenticates to Secret Manager with Application Default Credentials: the attached service account on Google Cloud, or `gcloud auth application-default login` on a workstation.
+
+## What changes with an external backend
+
+- `POST /providers` writes the provider row with `client_secret` set to NULL and stores the secret as `<prefix>-<provider id>` in the backend. If the backend write fails, the row is removed again and the request fails, so there is never a provider without its secret.
+- `PATCH /providers/{id}` with `client_secret` rotates the secret in the backend. Earlier versions are destroyed, so only the current value is readable and billed.
+- `PUT /providers/{id}` that omits `client_secret` leaves the stored secret unchanged. Clearing a secret is done by deleting the provider.
+- `DELETE /providers/...` deletes the secret from the backend as well as soft-deleting the row.
+- The OAuth exchange, token refresh, revocation and the provider health probe read the secret from the backend when they need it. It is never returned by the API.
+- If the backend has no secret for a provider (removed out of band), the provider still lists and can be inspected, but the exchange and refresh fail with `provider_secret_missing`, revocation destroys the credential locally only, and the health probe reports the provider `unhealthy` with the reason. An empty secret is never sent to the provider. To recover, set a new secret in place with `PATCH /providers/{id}` and a `client_secret`, or delete the provider and register it again; registering under the same name while the provider exists is rejected as a duplicate.
+- Secrets are keyed by provider ID, so renaming a provider never orphans its secret.
+
+Existing rows that still carry a column value keep working after you switch backends: the Broker reads the column when it is set and the backend only when it is NULL. New and updated providers go to the backend. Nothing moves automatically.
+
+Switching back from an external backend to `internal` is not transparent: providers registered while the backend was active have a NULL column, and the internal backend flags them as missing their secret rather than serving an empty one. Rotate each with `PATCH` to store the secret in the column, or keep the backend configured. Under the internal backend the Broker cannot delete Secret Manager secrets, so after rotating or deleting such a provider remove its old secret (`<prefix>-<provider id>`) yourself; cleaner is to delete the providers you intend to drop while the GCP backend is still configured.
+
+## Startup check
+
+With an external backend the Broker verifies at boot that it can do everything a registration does: it writes, reads and deletes a probe secret under the prefix (a fresh random id per start, removed on every exit path, alive for under a second) and refuses to start otherwise. A missing permission therefore shows up in the first log lines rather than on the first provider registration. The first log lines say which backend is active:
+
+```
+Provider client secrets: gcp-secret-manager (project my-project, prefix nexus-providers)
+```
+
+## Permissions on Google Cloud
+
+The Broker's service account needs `secretmanager.secrets.create` on the project, and on secrets under the prefix only get, delete and version add/access/list/destroy. Creating a secret is authorised against the project, so it cannot be restricted by name; the per-secret grant can be, with an IAM condition on `resource.name`, and it carries no IAM-policy permissions. The Terraform module at [`deploy/terraform/gcp/secret-backend`](https://github.com/Prescott-Data/nexus-framework/tree/main/deploy/terraform/gcp/secret-backend) sets up exactly that:
+
+```hcl
+module "nexus_secret_backend" {
+  source                 = "github.com/Prescott-Data/nexus-framework//deploy/terraform/gcp/secret-backend?ref=main"
+  project_id             = "my-project"
+  broker_service_account = google_service_account.broker.email
+  secret_prefix          = "nexus-providers"
+}
+```
+
+Secrets the Broker did not create, such as the ones holding `ENCRYPTION_KEY` or the database URL, stay unreadable to it.
+
+## Cost
+
+Secret Manager bills per active secret version per month and per ten thousand access operations. The Broker keeps one enabled version per provider and reads it only during OAuth exchanges, refreshes, revocations and the periodic health probe. For a deployment with tens of providers this is well under a dollar a month.
+
+## Other backends
+
+`SECRET_BACKEND` is a small interface (`pkg/vault/providers.Backend`: `Get`, `Set`, `Delete`, `Ping`). HashiCorp Vault and AWS Secrets Manager are tracked in [#63](https://github.com/Prescott-Data/nexus-framework/issues/63).

docs/infrastructure/deploying-nexus.md

@@ -162,6 +162,8 @@ Both keys must be identical across all instances of the same service. In Kuberne
 
 For Broker API key rotation, prefer `API_KEY_FILE` or `API_KEYS_FILE` backed by a mounted secret. The Broker reloads those files on `API_KEY_RELOAD_INTERVAL`, so updating the secret volume can add or remove accepted keys without restarting the pod.
 
+Provider `client_secret` values can be kept out of Postgres altogether with `SECRET_BACKEND=gcp-secret-manager`; see [Provider Secret Backends](../guides/provider-secret-backends.md).
+
 ## Health checks
 
 The Broker exposes `GET /health` and the Gateway exposes `GET /health`. Both return `200 OK` when the service is ready. Configure your load balancer to use these endpoints. The Sidecar also exposes `GET /health` plus Prometheus metrics at `GET /metrics`.

mkdocs.yml

@@ -114,6 +114,7 @@ nav:
       - Troubleshooting: guides/troubleshooting.md
       - Health Checks: guides/healthchecks.md
       - Security-as-Code: guides/security-as-code.md
+      - Provider Secret Backends: guides/provider-secret-backends.md
       - Deploying Nexus: infrastructure/deploying-nexus.md
   - Reference:
       - API Reference: reference/api.md
@@ -124,7 +125,7 @@ nav:
 
 
 extra:
-  version: "0.4.3"
+  version: "0.5.0"
   social:
     - icon: material/web
       link: https://developers.prescottdata.io

nexus-broker/Dockerfile

@@ -1,6 +1,6 @@
 # Multi-stage build for Dromos OAuth Broker
 
-FROM golang:1.25-alpine AS builder
+FROM golang:1.26-alpine AS builder
 WORKDIR /app
 
 # Enable modules and install build deps

nexus-broker/README.md

@@ -53,6 +53,11 @@ openssl rand -base64 32  # use output for STATE_KEY
 
 Keep ENCRYPTION_KEY and STATE_KEY constant; changing them breaks decrypting stored tokens.
 
+To keep provider `client_secret` values out of Postgres, set
+`SECRET_BACKEND=gcp-secret-manager` (plus `GCP_PROJECT_ID` outside Google
+Cloud). See `docs/guides/provider-secret-backends.md` and the Terraform module
+in `deploy/terraform/gcp/secret-backend`.
+
 For production API key rotation, mount a secret file and set `API_KEY_FILE`
 or `API_KEYS_FILE`. `API_KEY_FILE` may contain one key, while `API_KEYS_FILE`
 may contain comma- or newline-separated keys. The broker reloads these files at