security: content safety and prompt shields fail-open is a bypass risk

[imran-siddique]

Summary

Both Content Safety and Prompt Shields checks in inference-router/src/safety.rs are configured as fail-open — if the Azure AI Content Safety API is unreachable or returns an error, the request proceeds without safety checks.

Current Behavior (safety.rs)

// Content Safety check — fail-open
match check_content_safety(&text, &endpoint, &token).await {
    Ok(result) => { /* enforce if severity >= threshold */ }
    Err(e) => {
        warn!("Content Safety check failed, allowing request: {}", e);
        // REQUEST PROCEEDS WITHOUT SAFETY CHECK
    }
}

Same pattern for Prompt Shields.

Risk

An attacker who can cause the Content Safety endpoint to be unreachable (DNS manipulation, network partition, DoS on the endpoint) can bypass all safety checks. In a sandbox environment where the agent has shell access, this is a significant risk.

Proposal

1. Add a configurable fail_mode per safety check: fail-open (current default) or fail-closed
2. For production deployments, recommend fail-closed with a circuit breaker (after N failures, block all requests until the safety endpoint recovers)
3. Add a governance policy option: content_safety_fail_mode: closed in ClawSandbox CRD

AGT's agent-sre package has circuit breaker patterns that could be applied here.

[pallakatos]

Resolved in main (merge commit 8f90e58)

Added a circuit breaker with fail-open semantics and 60-second auto-reset cooldown.

Implementation: inference-router/src/safety.rs

• Circuit breaker opens after 5 consecutive Content Safety API failures
• When open: requests are allowed through (fail-open) with warning logs
• Auto-resets after 60 seconds to retry the safety service
• On successful API call, counter resets and circuit closes

// Lines 49-56
static SAFETY_FAILURES: AtomicU32 = AtomicU32::new(0);
static SAFETY_CIRCUIT_OPEN: AtomicU32 = AtomicU32::new(0);
static SAFETY_CIRCUIT_OPENED_AT: AtomicU32 = AtomicU32::new(0);
const CIRCUIT_BREAKER_THRESHOLD: u32 = 5;
const CIRCUIT_BREAKER_COOLDOWN_SECS: u32 = 60;

A downed safety service does not block legitimate traffic.

Key commits: 7e569af (circuit breaker), d660628 (fail-open + cooldown)