fix: re-curate compliance-057-ds oracle for v11.4.0 (ssoSettingsApi)

auditLoggingAppPlatform was added to grafana/grafana main on 2025-12-29,
a full year after v11.4.0 (2024-12-05). It doesn't exist in any release.

Re-curated to ssoSettingsApi feature flag which exists at v11.4.0 with
rich cross-component wiring: flag definition, SSO settings interfaces,
database persistence, REST API, OAuth gating, LDAP registration, RBAC.
10 oracle files across 4 architectural layers. Self-check: 1.0000.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: sjarmak

benchmarks/ccb_mcp_compliance/ccx-compliance-057-ds/instruction.md

@@ -1,34 +1,36 @@
-# Compliance Evidence Bundle: Audit Logging Feature in Grafana
+# Compliance Evidence Bundle: SSO Settings Feature in Grafana
 
 ## Scenario
 
 For a SOC 2 audit, your security team needs a compliance evidence bundle proving
-that Grafana's app platform audit logging control is implemented end-to-end. You
-need to trace the `auditLoggingAppPlatform` feature flag from its definition
-through the API server wiring to the audit backend infrastructure.
+that Grafana's SSO (Single Sign-On) settings control is implemented end-to-end.
+You need to trace the `ssoSettingsApi` feature flag from its definition through
+the SSO settings infrastructure, API endpoints, OAuth connector wiring, and
+authentication registration to show the control is properly gated and enforced.
 
 ## Your Task
 
-Find ALL files in `grafana/grafana` that form the audit logging control across
+Find ALL files in `grafana/grafana` that form the SSO settings control across
 these 4 layers:
 
 ### 1. Feature Flag Definition
-- The feature flag registry where `auditLoggingAppPlatform` is defined
-- Generated constants files (Go, TypeScript, CSV, JSON) that reference this flag
+- The feature flag registry where `ssoSettingsApi` is defined
+- Generated constants files (Go, TypeScript) that reference this flag
 
-### 2. Audit Infrastructure
-- The `Logger` interface definition for audit logging
-- The `Event` model/struct used for audit events
-- The `Policy` evaluation logic for audit decisions
-- The `Noop` implementation (no-op backend for when auditing is disabled)
+### 2. SSO Settings Infrastructure
+- The `Service` interface definition for SSO settings management
+- The `Reloadable` interface for live configuration reloading
+- The `SSOSettings` data model/struct with persistence annotations
+- The `SSOSettingsStore` database layer for SSO setting persistence
 
-### 3. API Server Wiring
-- The `APIGroupAuditor` interface definition
-- Policy aggregation logic that connects feature flags to audit backends
-- Service dependency injection that wires auditing into the API server
+### 3. API & Authentication Wiring
+- The REST API endpoint registration (`/api/v1/sso-settings`) with access control middleware
+- The `SocialService` provider that loads OAuth connectors when the flag is enabled
+- The authentication client registration that conditionally enables LDAP based on the flag
 
-### 4. Wire/DI Registration
-- Wire sets and generated wire code that register audit components
+### 4. Access Control & DI Registration
+- The access control evaluators (`EvalAuthenticationSettings`, `OauthSettingsEvaluator`) that gate SSO admin UI access
+- The `ProvideService` dependency injection function that wires the SSO settings store, API, fallback strategies, and reloadables
 
 ## Available Resources
 
@@ -45,21 +47,22 @@ Create a file at `/workspace/answer.json` with your findings:
   "files": [
     {"repo": "grafana/grafana", "path": "pkg/services/featuremgmt/registry.go"}
   ],
-  "text": "Comprehensive explanation of how the 4 layers connect: feature flag definition → audit infrastructure → API server wiring → DI registration."
+  "text": "Comprehensive explanation of how the 4 layers connect: feature flag definition → SSO settings infrastructure → API & authentication wiring → access control & DI registration."
 }
 ```
 
 **Important**: Use `grafana/grafana` as the exact `repo` identifier. Strip the
 `github.com/` prefix that Sourcegraph MCP tools return.
 
 **Deep Search hint**: This task requires synthesizing across feature management,
-API server infrastructure, and dependency injection layers. Deep Search is
-particularly well-suited for tracing these cross-cutting concerns.
+SSO settings infrastructure, OAuth connectors, and authentication registration
+layers. Deep Search is particularly well-suited for tracing these cross-cutting
+concerns.
 
 ## Evaluation
 
 Your answer will be scored on:
 - **File recall and precision**: Did you find the architecturally significant files across all 4 layers?
-- **Keyword coverage**: Does your answer reference the key interfaces and types (`auditLoggingAppPlatform`, `APIGroupAuditor`, `audit.Backend`, `NoopBackend`, `Logger`)?
+- **Keyword coverage**: Does your answer reference the key interfaces and types (`ssoSettingsApi`, `FlagSsoSettingsApi`, `SSOSettings`, `Reloadable`, `SSOSettingsStore`)?
 - **Provenance**: Does your answer cite the correct repos and directory paths?
 - **Rubric judge**: An LLM judge will assess evidence completeness, cross-component tracing, auditor actionability, and technical accuracy.

benchmarks/ccb_mcp_compliance/ccx-compliance-057-ds/tests/criteria.json

@@ -1,22 +1,22 @@
 [
   {
     "metric": "evidence_completeness",
-    "description": "Accurate: Does the answer identify all 4 layers of the audit control (feature flag definition, audit infrastructure, API server wiring, DI registration)? Attributed: Are all layers backed by specific file paths? Actionable: Could an auditor verify each layer independently?",
+    "description": "Accurate: Does the answer identify all 4 layers of the SSO settings control (feature flag definition, SSO settings infrastructure, API & authentication wiring, access control & DI registration)? Attributed: Are all layers backed by specific file paths? Actionable: Could an auditor verify each layer independently?",
     "max_score": 3
   },
   {
     "metric": "cross_component_tracing",
-    "description": "Accurate: Does the answer trace the connection between feature flag (auditLoggingAppPlatform) → audit backend (Logger/Event/Policy) → API server wiring (APIGroupAuditor, service.go)? Attributed: Are the connections between layers explicitly described? Actionable: Does the explanation show how enabling the feature flag activates the audit pipeline?",
+    "description": "Accurate: Does the answer trace the connection between feature flag (ssoSettingsApi / FlagSsoSettingsApi) → SSO settings infrastructure (Service, Store, Reloadable) → API endpoints & OAuth connector loading (api.go, socialimpl/service.go) → access control evaluators (ssoutils/utils.go)? Attributed: Are the connections between layers explicitly described? Actionable: Does the explanation show how enabling the feature flag activates the SSO settings pipeline?",
     "max_score": 3
   },
   {
     "metric": "actionable_for_auditor",
-    "description": "Accurate: Could a SOC 2 auditor use this evidence bundle to verify the audit logging control is implemented? Attributed: Does the bundle clearly state what each file's role is in the control? Actionable: Is the evidence organized so a non-developer auditor can follow the control chain?",
+    "description": "Accurate: Could a SOC 2 auditor use this evidence bundle to verify the SSO settings control is implemented? Attributed: Does the bundle clearly state what each file's role is in the control? Actionable: Is the evidence organized so a non-developer auditor can follow the control chain?",
     "max_score": 2
   },
   {
     "metric": "technical_accuracy",
-    "description": "Accurate: Are the cited file paths, interfaces (Logger, APIGroupAuditor), and struct names (Event, NoopBackend) technically correct? Attributed: Do the file paths match actual files in grafana/grafana v11.4.0? Actionable: Could a developer use these references to locate the exact code?",
+    "description": "Accurate: Are the cited file paths, interfaces (Service, Reloadable, Store), and type names (SSOSettings, SSOSettingsStore, FallbackStrategy) technically correct? Attributed: Do the file paths match actual files in grafana/grafana v11.4.0? Actionable: Could a developer use these references to locate the exact code?",
     "max_score": 2
   }
 ]

benchmarks/ccb_mcp_compliance/ccx-compliance-057-ds/tests/oracle_answer.json

@@ -2,24 +2,24 @@
   "files": [
     {"repo": "grafana/grafana", "path": "pkg/services/featuremgmt/registry.go"},
     {"repo": "grafana/grafana", "path": "pkg/services/featuremgmt/toggles_gen.go"},
-    {"repo": "grafana/grafana", "path": "pkg/apiserver/auditing/logger.go"},
-    {"repo": "grafana/grafana", "path": "pkg/apiserver/auditing/event.go"},
-    {"repo": "grafana/grafana", "path": "pkg/apiserver/auditing/policy.go"},
-    {"repo": "grafana/grafana", "path": "pkg/apiserver/auditing/noop.go"},
-    {"repo": "grafana/grafana", "path": "pkg/services/apiserver/builder/common.go"},
-    {"repo": "grafana/grafana", "path": "pkg/services/apiserver/builder/helper.go"},
-    {"repo": "grafana/grafana", "path": "pkg/services/apiserver/service.go"},
-    {"repo": "grafana/grafana", "path": "pkg/registry/apis/wireset.go"}
+    {"repo": "grafana/grafana", "path": "pkg/services/ssosettings/ssosettings.go"},
+    {"repo": "grafana/grafana", "path": "pkg/services/ssosettings/models/models.go"},
+    {"repo": "grafana/grafana", "path": "pkg/services/ssosettings/database/database.go"},
+    {"repo": "grafana/grafana", "path": "pkg/services/ssosettings/api/api.go"},
+    {"repo": "grafana/grafana", "path": "pkg/login/social/socialimpl/service.go"},
+    {"repo": "grafana/grafana", "path": "pkg/services/authn/authnimpl/registration.go"},
+    {"repo": "grafana/grafana", "path": "pkg/services/accesscontrol/ssoutils/utils.go"},
+    {"repo": "grafana/grafana", "path": "pkg/services/ssosettings/ssosettingsimpl/service.go"}
   ],
-  "text": "The Grafana audit logging control spans 4 architectural layers, forming a complete SOC 2 evidence chain. Layer 1 — Feature Flag Definition: pkg/services/featuremgmt/registry.go defines the auditLoggingAppPlatform feature flag in the feature toggle registry, and pkg/services/featuremgmt/toggles_gen.go contains the generated Go constant FlagAuditLoggingAppPlatform used throughout the codebase. Layer 2 — Audit Infrastructure: pkg/apiserver/auditing/logger.go defines the Logger interface that audit backends must implement, pkg/apiserver/auditing/event.go defines the Event struct modeling individual audit events, pkg/apiserver/auditing/policy.go implements the policy evaluation logic that determines which API operations should be audited, and pkg/apiserver/auditing/noop.go provides the NoopBackend implementation used when audit logging is disabled. Layer 3 — API Server Wiring: pkg/services/apiserver/builder/common.go defines the APIGroupAuditor interface that API group builders implement to declare their audit policies, pkg/services/apiserver/builder/helper.go contains the policy aggregation logic that combines per-group audit policies with the global audit.Backend, and pkg/services/apiserver/service.go is the main API server service that wires the audit backend into the server startup via dependency injection. Layer 4 — DI Registration: pkg/registry/apis/wireset.go contains the Wire provider set that registers the audit components in Grafana's dependency injection container, ensuring the audit backend is available throughout the application lifecycle.",
+  "text": "The Grafana SSO settings control spans 4 architectural layers, forming a complete SOC 2 evidence chain for the ssoSettingsApi feature flag. Layer 1 — Feature Flag Definition: pkg/services/featuremgmt/registry.go defines the ssoSettingsApi feature flag in the feature toggle registry with Stage GeneralAvailability and Expression true (enabled by default), owned by the identityAccessTeam. pkg/services/featuremgmt/toggles_gen.go contains the generated Go constant FlagSsoSettingsApi used throughout the codebase to gate SSO settings functionality. Layer 2 — SSO Settings Infrastructure: pkg/services/ssosettings/ssosettings.go defines three core interfaces: Service (List, GetForProvider, Upsert, Delete, Patch, RegisterReloadable, Reload), Reloadable (Reload, Validate for live configuration updates), and Store (Get, List, Upsert, Delete for persistence), plus FallbackStrategy for config-file backward compatibility. pkg/services/ssosettings/models/models.go defines the SSOSettings struct with Xorm persistence annotations (ID, Provider, Settings map, Created, Updated, IsDeleted, Source) and JSON marshal/unmarshal methods that convert between snake_case and camelCase. pkg/services/ssosettings/database/database.go implements SSOSettingsStore with ProvideStore constructor, providing Get, List, Upsert (with UUID generation), and Delete (soft delete) operations against the sso_setting table. Layer 3 — API & Authentication Wiring: pkg/services/ssosettings/api/api.go defines the REST API at /api/v1/sso-settings with RegisterAPIEndpoints that registers GET /, GET /:key, PUT /:key, DELETE /:key endpoints, all gated by access control middleware (ActionSettingsRead/ActionSettingsWrite with ScopeSettingsOAuth). pkg/login/social/socialimpl/service.go contains ProvideService for the SocialService which, when FlagSsoSettingsApi is enabled, loads all OAuth provider settings from the SSO settings service and creates OAuth connectors dynamically instead of reading from static config. pkg/services/authn/authnimpl/registration.go contains the authentication client registration logic that checks both FlagSsoSettingsApi and FlagSsoSettingsLDAP to conditionally register the LDAP authentication client even when LDAPAuthEnabled is false in static config. Layer 4 — Access Control & DI Registration: pkg/services/accesscontrol/ssoutils/utils.go defines EvalAuthenticationSettings (evaluates SAML write+read and LDAP status read permissions) and OauthSettingsEvaluator (dynamically generates evaluators for all configurable OAuth providers), which the admin navigation and API routing use to gate access to the SSO settings UI. pkg/services/ssosettings/ssosettingsimpl/service.go contains the ProvideService DI constructor that wires together the database store (via database.ProvideStore), fallback strategies (OAuthStrategy, LDAPStrategy, SAMLStrategy), the reloadables map, and conditionally registers the API endpoints when FlagSsoSettingsApi is enabled, completing the dependency injection chain.",
   "_metadata": {
     "oracle_type": "file_set_match",
     "discovery_method": "sourcegraph_keyword_search",
     "queries": [
-      "repo:^github.com/grafana/grafana$ auditLoggingAppPlatform",
-      "repo:^github.com/grafana/grafana$ file:pkg/apiserver/auditing/ Logger interface",
-      "repo:^github.com/grafana/grafana$ APIGroupAuditor",
-      "repo:^github.com/grafana/grafana$ file:pkg/registry/apis/ wireset"
+      "repo:^github.com/sg-benchmarks/grafana$ FlagSsoSettingsApi",
+      "repo:^github.com/sg-benchmarks/grafana$ file:ssosettings Service interface",
+      "repo:^github.com/sg-benchmarks/grafana$ file:ssosettings/api/api.go RegisterAPIEndpoints",
+      "repo:^github.com/sg-benchmarks/grafana$ file:ssoutils EvalAuthenticationSettings"
     ],
     "verified_at": "2026-02-21",
     "pinned_version": "v11.4.0"