Skip to content

Commit cc780d4

Browse files
authored
feat(helm): add BackendTLSPolicy support (NVIDIA#2728)
* feat(helm): add optional BackendTLSPolicy for e2e TLS Add grpcRoute.backendTLSPolicy values to optionally create a BackendTLSPolicy resource that enables end-to-end TLS between the Gateway proxy and the OpenShell gateway pod. The Gateway proxy terminates client-facing TLS and re-encrypts when connecting to the backend, validating the pod's certificate against a user-supplied CA ConfigMap. This removes the requirement to set server.disableTls=true when using HTTPS at the Gateway listener. Supported on OpenShift 4.22+ and other platforms with BackendTLSPolicy support in the Gateway API implementation. Update OpenShift and ingress documentation with e2e TLS instructions. Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * feat(helm,server): auto-create backend CA ConfigMap in certgen hook Extend the generate-certs command with --backend-ca-configmap-name and --backend-ca-source-secret flags. When BackendTLSPolicy is enabled, the certgen pre-install hook creates the CA ConfigMap automatically: - pkiInitJob mode (default): uses the CA from the generated PKI bundle. Fully automatic on first install. - cert-manager mode: reads ca.crt from the server TLS Secret. On first install the Secret does not exist yet (cert-manager reconciles after templates are applied), so the ConfigMap is created on the first helm upgrade. Logs a warning on the initial skip. The caCertificateConfigMapName value now defaults to <fullname>-backend-ca when empty, so users only need to set backendTLSPolicy.enabled=true. Update certgen RBAC to include configmaps get/create when the feature is enabled. Add CLI arg parsing tests for the new flags. Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * refactor(helm): add server.tls.enableMtls flag for mTLS control Replace automatic mTLS disabling based on BackendTLSPolicy with an explicit server.tls.enableMtls flag that defaults to true. The user is now responsible for setting this to false when using BackendTLSPolicy, as ingress proxies cannot present client certificates to backends. Updated: - values.yaml: Added server.tls.enableMtls (default true) - gateway-config.yaml: Check enableMtls instead of backendTLSPolicy - _gateway-workload.tpl: Check enableMtls for client CA mount - Tests: Updated to use enableMtls flag - Docs: Added enableMtls=false to BackendTLSPolicy examples - README: Document new flag and BackendTLSPolicy requirement Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * docs(helm): clarify cert-manager backend CA ConfigMap workflow Update documentation to explain the two-step install process required when using cert-manager with BackendTLSPolicy: 1. helm install - cert-manager issues the server certificate, but the certgen hook can't create the backend CA ConfigMap yet (cert-manager reconciles after templates are applied) 2. helm upgrade - certgen hook reads the CA from the cert-manager-issued certificate and creates the ConfigMap Previously, the docs said "created on first upgrade" without explaining why or that the feature won't work until then. The updated docs now: - Explain the timing issue (cert-manager reconciles after chart install) - Provide clear steps for the cert-manager workflow - Note that pkiInitJob (default) creates it immediately on install - Clarify that users must wait for the Certificate to be Ready before running the second upgrade Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * fix(docs): remove incorrect external hostname requirement for BackendTLSPolicy BackendTLSPolicy validates the backend certificate against the service FQDN (e.g., openshell.openshell.svc.cluster.local), not the external hostname. The external hostname only needs to be on the Gateway listener certificate for client-facing TLS. The default certManager.serverDnsNames already includes all required service FQDN variants, so no configuration is needed for BackendTLSPolicy to work. Fixed incorrect documentation that claimed: - "The server certificate SAN list must include the external hostname" - Users need to "configure certManager.serverDnsNames with the external hostname" Removed the unnecessary pkiInitJob.serverDnsNames override from the example and clarified that: - Gateway listener certificate needs the external hostname (for clients) - Backend certificate needs the service FQDN (for Gateway proxy) - The service FQDN is already in the defaults Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * docs: clarify ACME with LetsEncrypt reference Change all references from "ACME issuer" to "LetsEncrypt/ACME issuer" to help users understand that LetsEncrypt is the most common ACME provider and what ACME means in practice. Updated: - docs/kubernetes/managing-certificates.mdx - docs/kubernetes/openshift.mdx - deploy/helm/openshell/values.yaml - deploy/helm/openshell/README.md - deploy/helm/openshell/ci/values-openshift-route-cert-manager.yaml Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * docs(openshift): restructure end-to-end TLS options and clarify Gateway hostname Reorganize the OpenShift production deployment documentation: 1. Changed main section from "Production Deployments" to "Options for end-to-end TLS" for better clarity 2. Renamed subsections for consistency and clarity: - "End-to-end TLS using Gateway API and BackendTLSPolicy (OpenShift 4.22+)" - "End-to-end TLS using pass-through Route (all OpenShift versions)" 3. Clarified that the Gateway hostname is typically a wildcard: "typically a wildcard like *.openshell-ingress-gw.example.com" 4. Removed the recommendation to copy the cluster's wildcard certificate from openshift-ingress namespace, as this is not a recommended security best practice These changes make it clearer that users have two end-to-end TLS options and help them understand the typical naming pattern for Gateway hostnames. Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * feat(helm): eliminate two-stage install for BackendTLSPolicy with cert-manager When using BackendTLSPolicy with cert-manager, the certgen hook now polls for up to 90 seconds waiting for cert-manager to issue the TLS certificate before creating the backend CA ConfigMap. This eliminates the need for a second `helm upgrade` in most cases. The hook polls every 2 seconds with progress logging every 10 seconds. If cert-manager takes longer than 90 seconds, the hook times out gracefully and logs a warning, preserving the fallback to manual ConfigMap creation or a second upgrade. The Job's activeDeadlineSeconds is 120s, so the 90s timeout leaves 30s margin for ConfigMap creation and hook completion. Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * feat(helm): add configurable timeout for certgen hook Add `pkiInitJob.timeoutSeconds` Helm value (default 120) to control how long the certgen hook Job can run. When using cert-manager with BackendTLSPolicy, the hook polls for (timeoutSeconds - 30) seconds to leave margin for ConfigMap creation and cleanup. This allows users to increase the timeout for environments where cert-manager takes longer than 90 seconds to issue certificates, without requiring code changes. Example usage: ```yaml pkiInitJob: timeoutSeconds: 180 # Hook polls for 150 seconds ``` Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * docs(helm): document configurable certgen timeout Update documentation to mention the pkiInitJob.timeoutSeconds value and how it affects the cert-manager polling behavior when using BackendTLSPolicy. Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * feat(helm): add configurable failure behavior for certgen timeout Add `pkiInitJob.failOnTimeout` Helm value (default false) to control whether the certgen hook fails or succeeds when cert-manager does not issue a certificate within the polling timeout. When false (default), the hook succeeds with a warning and users can run `helm upgrade` after cert-manager issues the certificate to create the backend CA ConfigMap. This provides backwards-compatible behavior. When true, the hook fails immediately if the timeout is reached, providing clear feedback that BackendTLSPolicy is non-functional. This is useful for strict validation requirements where incomplete installs should fail fast. Example usage: ```yaml pkiInitJob: timeoutSeconds: 180 failOnTimeout: true # Fail install if cert-manager takes >150s ``` Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * feat(helm): change failOnTimeout default to true and add troubleshooting docs Change `pkiInitJob.failOnTimeout` default from false to true to provide immediate feedback when cert-manager does not issue certificates within the polling timeout. This prevents silent failures where BackendTLSPolicy is non-functional but the install appears to succeed. Add comprehensive troubleshooting section to docs/kubernetes/ingress.mdx documenting the specific error "TLS error: Secret is not supplied by SDS" that occurs when the backend CA ConfigMap is missing, with step-by-step resolution instructions. Updated comments in values.yaml to clearly document the default behavior and explain when administrators might see connectivity errors if they override the default to failOnTimeout=false. BREAKING CHANGE: pkiInitJob.failOnTimeout now defaults to true. Helm installs will fail if cert-manager takes longer than (timeoutSeconds - 30) seconds to issue certificates. To restore the old behavior of allowing installs to succeed with a warning, set `pkiInitJob.failOnTimeout=false`. Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * fix(helm): make cert-manager resources pre-install hooks to fix ordering Make Certificate and Issuer resources run as pre-install/pre-upgrade hooks with weight -30, before the certgen hook (weight -20). This fixes the chicken-and-egg problem where the certgen hook was waiting for Secrets created by Certificates that hadn't been created yet. **Hook ordering:** 1. Certificate and Issuer resources created (weight -30) 2. cert-manager issues certificates and creates Secrets 3. certgen hook runs (weight -20), finds Secrets, creates ConfigMap 4. Main resources (StatefulSet, Service, etc.) created Previously, the certgen pre-install hook would run before any resources were created, poll for a non-existent Secret, timeout, and fail. The Certificate resources would never get created because Helm waits for all pre-install hooks to succeed before creating main resources. This fix allows single-stage installs to work reliably as long as cert-manager can issue certificates within the polling timeout. Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * feat(helm): add validation to prevent enableMtls with BackendTLSPolicy Add Helm chart validation that fails the install if both server.tls.enableMtls=true and grpcRoute.backendTLSPolicy.enabled=true are set, since this is an invalid configuration. BackendTLSPolicy requires mTLS to be disabled because the Gateway proxy cannot present client certificates to the backend. This validation provides immediate, clear feedback at install time rather than allowing the misconfiguration to be discovered through runtime errors. Example error message: ``` Error: grpcRoute.backendTLSPolicy requires mTLS to be disabled because the Gateway proxy cannot present client certificates to the backend; set server.tls.enableMtls=false ``` Also updated documentation to mention this validation check. Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * docs(helm): clarify pkiInitJob.timeoutSeconds polling behavior Improve documentation to clearly explain that pkiInitJob.timeoutSeconds controls the Job deadline, but the actual polling timeout is (timeoutSeconds - 30) to reserve 30 seconds for ConfigMap creation and cleanup. Added concrete example: "timeoutSeconds=180 allows 150 seconds of polling" to make the relationship explicit and avoid confusion where users might expect the hook to poll for the full timeout value. Updated both values.yaml inline comments and ingress.mdx documentation for consistency. Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * docs(openshift): remove outdated two-stage install instructions Update OpenShift documentation to reflect that single-stage installs now work with cert-manager and BackendTLSPolicy. The Certificate resources run as pre-install hooks (weight -30) before certgen (weight -20), allowing the hook to poll for and find the issued certificates. Removed the outdated two-step process: 1. helm install (cert-manager issues cert, hook logs warning) 2. helm upgrade (hook creates ConfigMap) Replaced with current single-stage behavior: - Certificate resources created as pre-install hooks - certgen hook polls for up to 90 seconds (configurable) - Single helm install succeeds in most cases - Fails fast by default if timeout reached This brings openshift.mdx in line with the already-updated ingress.mdx documentation. Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> * fix(helm): make pkiInitJob.timeoutSeconds the actual polling duration The timeout value now represents the actual polling time that users experience when waiting for cert-manager to issue certificates. The Job activeDeadlineSeconds is set to (timeoutSeconds + 30) to allow buffer time for ConfigMap creation and cleanup. Previously, the hook polled for (timeoutSeconds - 30) seconds, which was confusing when users set timeoutSeconds=180 and only got 150 seconds of actual polling. Updated documentation in values.yaml, ingress.mdx, and openshift.mdx to reflect the clearer behavior. Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com> * docs(helm): update values.yaml and README with correct polling duration Updated the caCertificateConfigMapName description to reflect that the hook polls for exactly pkiInitJob.timeoutSeconds seconds, not (timeoutSeconds - 30) seconds. Regenerated README.md with helm-docs. Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com> * docs(kubernetes): add OIDC configuration to helm install and CLI examples Updated all helm install and openshell gateway add examples in ingress.mdx and openshift.mdx to include OIDC issuer and audience configuration. Examples now use concrete placeholder values: - OIDC issuer: https://keycloak.example.com/realms/openshell - OIDC audience: openshell-cli - Hostname: gateway.example.com - ClusterIssuer: letsencrypt-prod This makes it clearer how to configure OIDC authentication, which is required when using BackendTLSPolicy or HTTPS termination since the Gateway proxy cannot present client certificates to the backend. Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com> * docs(kubernetes): explicitly list OIDC client ID in gateway add examples Added --oidc-client-id openshell-cli to all openshell gateway add commands in ingress.mdx and openshift.mdx, making the default client ID explicit in the examples even though it's the CLI default. This improves clarity and helps users understand the complete OIDC configuration needed for gateway registration. Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com> * fix(helm): address PR review feedback for BackendTLSPolicy - Read backend CA from the authoritative server Secret instead of the in-memory PKI bundle so enabling BackendTLSPolicy on an existing release uses the CA that actually signed the server certificate. - Reconcile the backend CA ConfigMap on every hook run (compare and update) instead of skipping when it already exists, so CA rotations propagate automatically. - Remove hook annotations from cert-manager Issuer/Certificate resources so they remain regular release objects managed by Helm lifecycle. Split the cert-manager backend CA ConfigMap creation into a separate post-install/post-upgrade hook Job that polls after cert-manager Certificate resources are applied. - Update architecture/gateway.md, docs/reference/gateway-config.mdx, debug-openshell-cluster skill, and helm-dev-environment skill with BackendTLSPolicy, backend CA ConfigMap, enableMtls, and timeout documentation. Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com> Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * fix(docs): resolve markdown lint errors in helm README and kubernetes docs Escape inline HTML angle brackets in README.md template placeholders, remove trailing spaces, and add blank lines around fenced code blocks in numbered lists. Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * Update docs * fix(helm): escape inline HTML in values.yaml descriptions and sync mise lockfile Wrap `<fullname>` and `<namespace>` template placeholders in backticks so markdownlint does not flag them as inline HTML (MD033). Regenerate mise.lock to match current mise.toml after rebase onto main. Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * Run 'mise lock' * fix: align mise.lock with CI mise version output The lockfile was regenerated locally with mise 2026.8.10 which resolves uv Linux artifacts to gnu variants and adds provenance_verified fields, but CI uses v2026.4.25 which produces musl variants without those fields. Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> * fix(docs): correct cert-manager hook ordering and clientCaSecretName comment Update ingress.mdx and openshift.mdx to describe Certificate resources as regular release objects with a post-install/post-upgrade Job, matching the current implementation and architecture/gateway.md. Fix values.yaml clientCaSecretName comment to state that "" disables client certificate verification, matching the helper and access-control docs. Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> --------- Signed-off-by: Brandon Squizzato <bsquizza@redhat.com> Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com> Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com>
1 parent 8d19308 commit cc780d4

20 files changed

Lines changed: 735 additions & 36 deletions

File tree

‎.agents/skills/helm-dev-environment/SKILL.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -225,6 +225,35 @@ Envoy Gateway is already installed by Skaffold (the `envoy-gateway` Helm release
225225
service for the proxy; klipper-lb binds it to hostPort 80, reachable via the
226226
`8080:80` load balancer port mapping.
227227

228+
### BackendTLSPolicy (end-to-end TLS)
229+
230+
To enable end-to-end TLS between the Gateway proxy and the gateway pod, add
231+
BackendTLSPolicy values to the Helm install:
232+
233+
```bash
234+
helm upgrade --install openshell deploy/helm/openshell \
235+
--set grpcRoute.enabled=true \
236+
--set grpcRoute.backendTLSPolicy.enabled=true \
237+
--set server.tls.enableMtls=false \
238+
...
239+
```
240+
241+
This requires `server.tls.enableMtls=false` because ingress proxies cannot
242+
present client certificates to the backend. The certgen hook creates a backend
243+
CA ConfigMap from the server Secret's `ca.crt` key. With cert-manager, a
244+
separate post-install Job polls for the cert-manager-issued certificate (up to
245+
`pkiInitJob.timeoutSeconds`); with built-in PKI the ConfigMap is created in the
246+
same pre-install hook. The ConfigMap is reconciled on every upgrade so CA
247+
rotations propagate automatically.
248+
249+
Key Helm values:
250+
- `grpcRoute.backendTLSPolicy.enabled`: create the BackendTLSPolicy resource
251+
- `grpcRoute.backendTLSPolicy.caCertificateConfigMapName`: override ConfigMap name
252+
- `grpcRoute.backendTLSPolicy.hostname`: override backend validation hostname
253+
- `server.tls.enableMtls`: must be `false` for BackendTLSPolicy
254+
- `pkiInitJob.timeoutSeconds`: polling duration for cert-manager mode
255+
- `pkiInitJob.failOnTimeout`: fail install if cert-manager times out
256+
228257
### Keycloak OIDC
229258

230259
One-time setup — only needed once per cluster lifetime:

‎architecture/gateway.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -762,6 +762,25 @@ requested present -> generate and write. This guards continuity across restarts
762762
and upgrades while still recovering cleanly if an operator deletes everything
763763
and starts over.
764764

765+
When `grpcRoute.backendTLSPolicy.enabled=true`, the certgen hook also creates a
766+
`ConfigMap` containing the CA certificate (`ca.crt`) used by the Gateway proxy
767+
to validate the backend pod's TLS certificate. The CA is always read from the
768+
authoritative server Secret (not the in-memory bundle) so that enabling
769+
BackendTLSPolicy on an existing release uses the CA that actually signed the
770+
server certificate. The ConfigMap is reconciled on every hook run: if the CA
771+
changes (rotation, re-issue), the ConfigMap is updated in place. In built-in PKI
772+
mode the ConfigMap is created in the same pre-install hook. In cert-manager
773+
mode, a separate post-install/post-upgrade hook Job polls for the cert-manager-
774+
issued server Secret and then creates or updates the ConfigMap, because
775+
cert-manager Certificate resources are regular release objects applied after
776+
pre-install hooks.
777+
778+
The `server.tls.enableMtls` value controls whether the gateway requires client
779+
certificates. When `enableMtls` is `false`, the gateway runs HTTPS-only without
780+
client certificate verification (use OIDC for identity instead). BackendTLSPolicy
781+
requires `enableMtls=false` because the ingress proxy cannot present client
782+
certificates to the backend.
783+
765784
Operators who manage TLS PKI with cert-manager enable `certManager.enabled`;
766785
cert-manager takes precedence over built-in TLS generation and the chart still
767786
renders the JWT-only hook. Operators who pre-create all TLS and JWT Secrets can

‎crates/openshell-server/src/certgen.rs‎

Lines changed: 194 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
2525
use clap::Args;
2626
use k8s_openapi::ByteString;
27-
use k8s_openapi::api::core::v1::Secret;
27+
use k8s_openapi::api::core::v1::{ConfigMap, Secret};
2828
use kube::Client;
2929
use kube::api::{Api, ObjectMeta, PostParams};
3030
use miette::{IntoDiagnostic, Result, WrapErr};
@@ -78,6 +78,33 @@ pub struct CertgenArgs {
7878
/// For local debugging.
7979
#[arg(long)]
8080
dry_run: bool,
81+
82+
/// Name of a `ConfigMap` to create containing the CA certificate (key: ca.crt)
83+
/// for `BackendTLSPolicy` backend validation. The CA is always read from the
84+
/// authoritative server Secret: --server-secret-name in full PKI mode,
85+
/// --backend-ca-source-secret in --jwt-only mode.
86+
#[arg(long, value_name = "NAME")]
87+
backend_ca_configmap_name: Option<String>,
88+
89+
/// Name of an existing `Secret` containing a ca.crt key to populate the
90+
/// backend CA `ConfigMap` from. Required with --jwt-only when
91+
/// --backend-ca-configmap-name is set (typically the server TLS `Secret`
92+
/// created by cert-manager).
93+
#[arg(long, value_name = "NAME", requires = "backend_ca_configmap_name")]
94+
backend_ca_source_secret: Option<String>,
95+
96+
/// Maximum time in seconds to poll for the backend CA source `Secret` when
97+
/// using cert-manager. Defaults to 90 seconds. The Helm chart sets this to
98+
/// (Job activeDeadlineSeconds - 30) to leave margin for `ConfigMap` creation.
99+
#[arg(long, value_name = "SECONDS", default_value = "90")]
100+
backend_ca_poll_timeout_seconds: u64,
101+
102+
/// Fail with an error if the backend CA source `Secret` is not found within
103+
/// the polling timeout. When false (default), the hook succeeds with a warning
104+
/// and the `ConfigMap` is not created. The Helm chart sets this based on
105+
/// pkiInitJob.failOnTimeout.
106+
#[arg(long)]
107+
backend_ca_fail_on_timeout: bool,
81108
}
82109

83110
pub async fn run(args: CertgenArgs) -> Result<()> {
@@ -97,7 +124,13 @@ pub async fn run(args: CertgenArgs) -> Result<()> {
97124
run_local(dir, &args.server_sans)
98125
} else {
99126
let bundle = generate_pki(&args.server_sans)?;
100-
run_kubernetes(&args, &bundle).await
127+
run_kubernetes(&args, &bundle).await?;
128+
129+
if let Some(ref cm_name) = args.backend_ca_configmap_name {
130+
create_backend_ca_configmap_if_needed(&args, cm_name).await?;
131+
}
132+
133+
Ok(())
101134
}
102135
}
103136

@@ -293,6 +326,165 @@ async fn create_tls_secrets(
293326
Ok(())
294327
}
295328

329+
fn extract_ca_from_secret(secret: &Secret, name: &str) -> Result<String> {
330+
let data = secret
331+
.data
332+
.as_ref()
333+
.ok_or_else(|| miette::miette!("secret {name} has no data"))?;
334+
let ca = data
335+
.get("ca.crt")
336+
.ok_or_else(|| miette::miette!("secret {name} has no ca.crt key"))?;
337+
String::from_utf8(ca.0.clone())
338+
.into_diagnostic()
339+
.wrap_err("ca.crt is not valid UTF-8")
340+
}
341+
342+
async fn create_backend_ca_configmap_if_needed(
343+
args: &CertgenArgs,
344+
configmap_name: &str,
345+
) -> Result<()> {
346+
let namespace = args
347+
.namespace
348+
.as_deref()
349+
.ok_or_else(|| miette::miette!("--namespace is required (or set POD_NAMESPACE)"))?;
350+
351+
let client = Client::try_default()
352+
.await
353+
.into_diagnostic()
354+
.wrap_err("failed to construct Kubernetes client for backend CA ConfigMap")?;
355+
let secret_api: Api<Secret> = Api::namespaced(client.clone(), namespace);
356+
let cm_api: Api<ConfigMap> = Api::namespaced(client, namespace);
357+
358+
// Resolve the CA from the authoritative server Secret rather than the
359+
// in-memory bundle so upgrades that enable BackendTLSPolicy on an
360+
// existing release use the CA that actually signed the server cert.
361+
let ca_pem = if let Some(source_secret) = &args.backend_ca_source_secret {
362+
let poll_timeout = std::time::Duration::from_secs(args.backend_ca_poll_timeout_seconds);
363+
let poll_interval = std::time::Duration::from_secs(2);
364+
let start = std::time::Instant::now();
365+
366+
let secret = loop {
367+
match secret_api
368+
.get_opt(source_secret)
369+
.await
370+
.into_diagnostic()
371+
.wrap_err_with(|| format!("failed to read secret {source_secret}"))?
372+
{
373+
Some(secret) => break secret,
374+
None if start.elapsed() >= poll_timeout => {
375+
let msg = format!(
376+
"Backend CA source secret {source_secret} not found after {timeout_secs}s; \
377+
ConfigMap {configmap_name} not created. This is expected if cert-manager \
378+
is still issuing the certificate.",
379+
timeout_secs = poll_timeout.as_secs()
380+
);
381+
if args.backend_ca_fail_on_timeout {
382+
return Err(miette::miette!(
383+
"{msg} Install failed due to --backend-ca-fail-on-timeout."
384+
));
385+
}
386+
warn!(
387+
secret = %source_secret,
388+
configmap = %configmap_name,
389+
timeout_secs = poll_timeout.as_secs(),
390+
"{msg} Run helm upgrade after the TLS secret exists or the BackendTLSPolicy \
391+
will remain non-functional until the ConfigMap is created manually."
392+
);
393+
return Ok(());
394+
}
395+
None => {
396+
if start.elapsed().as_secs().is_multiple_of(10) {
397+
info!(
398+
secret = %source_secret,
399+
elapsed_secs = start.elapsed().as_secs(),
400+
"Waiting for cert-manager to issue TLS certificate..."
401+
);
402+
}
403+
tokio::time::sleep(poll_interval).await;
404+
}
405+
}
406+
};
407+
408+
info!(
409+
secret = %source_secret,
410+
elapsed_secs = start.elapsed().as_secs(),
411+
"cert-manager TLS certificate found."
412+
);
413+
extract_ca_from_secret(&secret, source_secret)?
414+
} else if let Some(server_secret) = &args.server_secret_name {
415+
let secret = secret_api
416+
.get(server_secret)
417+
.await
418+
.into_diagnostic()
419+
.wrap_err_with(|| format!("failed to read server secret {server_secret}"))?;
420+
extract_ca_from_secret(&secret, server_secret)?
421+
} else {
422+
return Err(miette::miette!(
423+
"--backend-ca-source-secret or --server-secret-name is required \
424+
with --backend-ca-configmap-name"
425+
));
426+
};
427+
428+
// Reconcile: create or update so the backend CA stays current across
429+
// CA rotations and upgrades.
430+
if let Some(existing) = cm_api
431+
.get_opt(configmap_name)
432+
.await
433+
.into_diagnostic()
434+
.wrap_err_with(|| format!("failed to read configmap {configmap_name}"))?
435+
{
436+
let up_to_date = existing
437+
.data
438+
.as_ref()
439+
.and_then(|d| d.get("ca.crt"))
440+
.map(String::as_str)
441+
== Some(&ca_pem);
442+
if up_to_date {
443+
info!(
444+
namespace = %namespace,
445+
configmap = %configmap_name,
446+
"Backend CA ConfigMap is up-to-date, skipping."
447+
);
448+
return Ok(());
449+
}
450+
let mut updated = existing;
451+
updated.data = Some(BTreeMap::from([("ca.crt".to_string(), ca_pem)]));
452+
cm_api
453+
.replace(configmap_name, &PostParams::default(), &updated)
454+
.await
455+
.into_diagnostic()
456+
.wrap_err_with(|| format!("failed to update configmap {configmap_name}"))?;
457+
info!(
458+
namespace = %namespace,
459+
configmap = %configmap_name,
460+
"Backend CA ConfigMap updated with current CA."
461+
);
462+
return Ok(());
463+
}
464+
465+
let configmap = ConfigMap {
466+
metadata: ObjectMeta {
467+
name: Some(configmap_name.to_string()),
468+
..Default::default()
469+
},
470+
data: Some(BTreeMap::from([("ca.crt".to_string(), ca_pem)])),
471+
..Default::default()
472+
};
473+
474+
cm_api
475+
.create(&PostParams::default(), &configmap)
476+
.await
477+
.into_diagnostic()
478+
.wrap_err_with(|| format!("failed to create configmap {configmap_name}"))?;
479+
480+
info!(
481+
namespace = %namespace,
482+
configmap = %configmap_name,
483+
"Backend CA ConfigMap created."
484+
);
485+
Ok(())
486+
}
487+
296488
fn tls_secret(name: &str, crt_pem: &str, key_pem: &str, ca_pem: &str) -> Secret {
297489
let mut data = BTreeMap::new();
298490
data.insert(

‎crates/openshell-server/src/cli.rs‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2314,6 +2314,59 @@ mod tests {
23142314
.expect("complete package-generated TLS paths may not exist before certificate generation");
23152315
}
23162316

2317+
#[test]
2318+
fn generate_certs_backend_ca_configmap_flags_parse() {
2319+
let _lock = ENV_LOCK
2320+
.lock()
2321+
.unwrap_or_else(std::sync::PoisonError::into_inner);
2322+
let _g1 = EnvVarGuard::remove("OPENSHELL_DB_URL");
2323+
let _g2 = EnvVarGuard::remove("POD_NAMESPACE");
2324+
2325+
let cli = Cli::try_parse_from([
2326+
"openshell-gateway",
2327+
"generate-certs",
2328+
"--namespace",
2329+
"openshell",
2330+
"--jwt-only",
2331+
"--jwt-secret-name",
2332+
"openshell-jwt-keys",
2333+
"--backend-ca-configmap-name",
2334+
"openshell-backend-ca",
2335+
"--backend-ca-source-secret",
2336+
"openshell-server-tls",
2337+
])
2338+
.expect("backend CA ConfigMap flags should parse with --jwt-only");
2339+
2340+
assert!(matches!(
2341+
cli.command,
2342+
Some(super::Commands::GenerateCerts(_))
2343+
));
2344+
}
2345+
2346+
#[test]
2347+
fn generate_certs_backend_ca_source_secret_requires_configmap_name() {
2348+
let _lock = ENV_LOCK
2349+
.lock()
2350+
.unwrap_or_else(std::sync::PoisonError::into_inner);
2351+
let _g1 = EnvVarGuard::remove("OPENSHELL_DB_URL");
2352+
let _g2 = EnvVarGuard::remove("POD_NAMESPACE");
2353+
2354+
let err = Cli::try_parse_from([
2355+
"openshell-gateway",
2356+
"generate-certs",
2357+
"--namespace",
2358+
"openshell",
2359+
"--jwt-only",
2360+
"--jwt-secret-name",
2361+
"openshell-jwt-keys",
2362+
"--backend-ca-source-secret",
2363+
"openshell-server-tls",
2364+
])
2365+
.expect_err("--backend-ca-source-secret should require --backend-ca-configmap-name");
2366+
2367+
assert_eq!(err.kind(), clap::error::ErrorKind::MissingRequiredArgument);
2368+
}
2369+
23172370
#[test]
23182371
fn bare_invocation_with_no_db_url_parses_for_runtime_defaults() {
23192372
// db_url is Option<String> at the clap level so subcommand parsing

0 commit comments

Comments
 (0)