feat(sentinel): triage groups with judge, open the list on what is relevant

sentinel/README.md

@@ -82,15 +82,35 @@ and the logs of that trace.
 read the checkout mapped to the failing worker, and you can write to it at any
 time; a message lands in the turn that is already running. When it has a
 cause it records one by calling `sentinel::diagnosis::record`, which is the
-only write its policy allows — everything else it can reach is a read, and the
-engine's raw telemetry is not on the list at all. Each recording is a version;
+only write to this worker its policy allows. Besides reads of the code and
+the evidence, it can reach GitHub and the web, writes included (a PR merge,
+a POST); the engine's raw telemetry is not on the list at all. Each recording is a version;
 the most recent one stands and the earlier ones stay, so the same failure
 diagnosed twice can be compared.
 
 **Open in chat** does the same thing without running anything: the session is
 created with the evidence already in the transcript and waits for you to
 speak.
 
+Each group is also **triaged** once, five minutes after it is first seen,
+into `defect`, `caller_error`, `transient`, `environment` or `test_traffic`,
+and the label rides on `sentinel::groups::list` and `::get` as `triage`. A
+"function not found" whose function is registered by then is decided
+without a model — a restart when it was brief, the environment when it was
+not; everything else goes to [`judge`](../judge/) in batches. The judge sees
+the group as it was stored, so already redacted. Without `judge` deployed the
+groups simply stay untriaged, and a failing judge is left alone for five
+minutes. A label is a hint for ordering and filtering, never a state change.
+Switch it off, or change the wait, under **Triage** in the page's settings;
+which judge answers is chosen in the `judge` worker's own settings.
+
+The list opens on **Relevant**: defects, regressions, groups not triaged yet,
+and caller errors or environment problems that repeat (20 or more
+occurrences across an hour or more), since a program repeating a failing call
+needs a fix even when its message is a polite refusal. **Noise** holds the
+rest, ordered by kind, and each side shows its count, so nothing is more than
+a click away. `sentinel::groups::list` takes the same choice as `relevance`.
+
 Resolving and ignoring are yours. An ignore can last forever, for a number of
 further occurrences, or until the worker version changes — and the counters
 keep running either way, so an ignored group still tells you how often it
@@ -130,6 +150,9 @@ retention:
   resolved_ttl_days: 90
 investigation:
   model: ""                             # catalog id an investigation opens with; each run may pick another
+triage:
+  enabled: true
+  delay_ms: 300000                      # wait this long after first seen; a restart registers what it was missing
 projects:                               # where a worker's source lives on this machine (formerly `repositories`, still read)
   - id: workers
     path: /home/me/workspaces/workers

sentinel/iii.worker.yaml

@@ -37,6 +37,9 @@ config:
     cron: "0 0 3 * * *"
   investigation:
     model: ""
+  triage:
+    enabled: true
+    delay_ms: 300000
   projects: []
   service_aliases: {}
   database: primary

sentinel/src/config.rs

@@ -183,6 +183,27 @@ pub struct InvestigationConfigV1 {
     pub provider: Option<String>,
 }
 
+/// Sorting the noise from the defects. A group waits `delay_ms` after it is
+/// first seen — long enough for a restart to finish registering what it was
+/// missing — and is then classified once: by a deterministic check when one
+/// applies, otherwise by `judge::evaluate`. Without a judge deployed the
+/// groups simply stay untriaged.
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
+#[serde(deny_unknown_fields, default)]
+pub struct TriageConfigV1 {
+    pub enabled: bool,
+    pub delay_ms: u64,
+}
+
+impl Default for TriageConfigV1 {
+    fn default() -> Self {
+        Self {
+            enabled: true,
+            delay_ms: 300_000,
+        }
+    }
+}
+
 /// Where a worker's source lives on this machine. A worker with no repository
 /// is still grouped, still investigated — the agent just works from the
 /// evidence alone and says so.
@@ -222,6 +243,7 @@ pub struct WorkerConfig {
     pub evidence: EvidenceConfigV1,
     pub retention: RetentionConfigV1,
     pub investigation: InvestigationConfigV1,
+    pub triage: TriageConfigV1,
     /// Where each worker's source lives, so an investigation can read it.
     pub projects: Vec<RepositoryConfigV1>,
     /// The name `projects` had until 2026-09. Still read, so a value stored
@@ -256,6 +278,7 @@ impl Default for WorkerConfig {
             evidence: EvidenceConfigV1::default(),
             retention: RetentionConfigV1::default(),
             investigation: InvestigationConfigV1::default(),
+            triage: TriageConfigV1::default(),
             projects: Vec::new(),
             former_repositories: None,
             service_aliases: BTreeMap::new(),

sentinel/src/contract.rs

@@ -119,8 +119,73 @@ pub enum GroupChangeReasonV1 {
     Investigating,
 }
 
+/// What kind of failure a group is, as triage reads it.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
+#[serde(rename_all = "snake_case")]
+pub enum TriageKindV1 {
+    /// A bug the reporting worker's maintainer has to fix in code.
+    Defect,
+    /// The worker rejected bad input and said so: working as designed.
+    CallerError,
+    /// A dependency starting, restarting or timing out; it went away.
+    Transient,
+    /// Local setup: credentials, tokens, files, versions, a missing worker.
+    Environment,
+    /// Tests, probes and deliberately fake identifiers.
+    TestTraffic,
+}
+
+impl TriageKindV1 {
+    pub fn as_str(self) -> &'static str {
+        match self {
+            Self::Defect => "defect",
+            Self::CallerError => "caller_error",
+            Self::Transient => "transient",
+            Self::Environment => "environment",
+            Self::TestTraffic => "test_traffic",
+        }
+    }
+}
+
+/// Which side of triage the list shows.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
+#[serde(rename_all = "snake_case")]
+pub enum RelevanceV1 {
+    /// What needs a person: defects, persistent caller errors and
+    /// environment problems, regressions, and anything not triaged yet.
+    Relevant,
+    /// Everything else.
+    Noise,
+}
+
+/// Who decided a triage.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
+#[serde(rename_all = "snake_case")]
+pub enum TriageSourceV1 {
+    /// A deterministic check: a missing function that is registered by the
+    /// time the group is looked at.
+    Rule,
+    /// `judge::evaluate`.
+    Judge,
+}
+
+/// A hint for ordering and filtering the list. It never moves a group:
+/// resolving and ignoring stay human decisions.
+#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
+#[serde(deny_unknown_fields)]
+pub struct GroupTriageV1 {
+    pub kind: TriageKindV1,
+    /// Between 0 and 1, as the judge reports it; 1 for a rule.
+    pub confidence: f64,
+    pub source: TriageSourceV1,
+    /// The judge model that answered.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub model: Option<String>,
+    pub at_ms: i64,
+}
+
 /// A group as the list shows it.
-#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
+#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
 #[serde(deny_unknown_fields)]
 pub struct GroupSummaryV1 {
     pub id: String,
@@ -160,6 +225,11 @@ pub struct GroupSummaryV1 {
     pub resolved_version: Option<String>,
     #[serde(default, skip_serializing_if = "std::ops::Not::not")]
     pub resolve_until_version_change: bool,
+    /// Absent until the group has been triaged.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub triage: Option<GroupTriageV1>,
+    /// Whether the default list shows it; see [`RelevanceV1::Relevant`].
+    pub relevant: bool,
 }
 
 #[derive(Debug, Clone, Default, Deserialize, JsonSchema)]
@@ -176,6 +246,9 @@ pub struct GroupsListRequestV1 {
     /// Matches the title, the message sample and the function id.
     #[serde(default)]
     pub search: Option<String>,
+    /// Absent lists both sides.
+    #[serde(default)]
+    pub relevance: Option<RelevanceV1>,
     #[serde(default)]
     pub offset: Option<u32>,
     #[serde(default)]
@@ -192,6 +265,9 @@ pub struct GroupsListRequestV1 {
 pub struct GroupsListResponseV1 {
     pub groups: Vec<GroupSummaryV1>,
     pub total: u64,
+    /// Both sides under the same filters, whatever `relevance` asked for.
+    pub relevant_total: u64,
+    pub noise_total: u64,
 }
 
 #[derive(Debug, Clone, Default, Deserialize, JsonSchema)]

sentinel/src/functions.rs

@@ -748,9 +748,12 @@ pub fn catalog() -> Vec<FunctionSpec> {
 
 /// Function ids an investigation session may call. There is no approval gate
 /// in this design, so this list is the whole permission model: it holds the
-/// reads an investigation needs and exactly one write, which is this worker's
-/// own record of the diagnosis.
-pub const INVESTIGATION_ALLOW: [&str; 11] = [
+/// reads an investigation needs, this worker's own record of the diagnosis,
+/// and GitHub and the web. Those two are reached on purpose — an issue, a PR
+/// or a page can explain a failure — and they are the one place the
+/// read-only rule does not hold: `github::*` can merge and create, and
+/// `web::fetch` can POST.
+pub const INVESTIGATION_ALLOW: [&str; 13] = [
     "coder::info",
     "coder::read-file",
     "coder::search",
@@ -762,17 +765,18 @@ pub const INVESTIGATION_ALLOW: [&str; 11] = [
     TRACE_GET_ID,
     LOGS_LIST_ID,
     DIAGNOSIS_RECORD_ID,
+    "github::*",
+    "web::*",
 ];
 
 /// Denied outright. Deny wins over allow in the harness policy, so a function
 /// added to this worker later is refused until somebody decides otherwise.
-pub const INVESTIGATION_DENY: [&str; 21] = [
+pub const INVESTIGATION_DENY: [&str; 20] = [
     "shell::*",
     "state::*",
     "queue::*",
     "worktree::*",
     "harness::*",
-    "github::*",
     "configuration::*",
     "storage::*",
     "database::*",

sentinel/src/iii_runtime/mod.rs

@@ -22,6 +22,7 @@ use crate::ingest::{ring::PhantomRing, CheckoutVersions, IngestJob, Telemetry, T
 use crate::registry::{EngineRegistry, FunctionEntry, WorkerEntry};
 use crate::service::TraceAvailability;
 use crate::store::{Db, NamedRow, Statement, StepResult};
+use crate::triage::{self, Judge};
 use crate::{SentinelError, WorkerConfig};
 
 /// The tag that keeps this worker's own calls out of the trace views.
@@ -47,6 +48,16 @@ impl Runtime {
     /// Call a function with this worker's own traffic marked hidden, and the
     /// resulting trace remembered so its tick is dropped rather than ingested.
     async fn call(&self, function_id: &str, payload: Value) -> Result<Value, SentinelError> {
+        self.call_within(function_id, payload, CALL_TIMEOUT_MS)
+            .await
+    }
+
+    async fn call_within(
+        &self,
+        function_id: &str,
+        payload: Value,
+        timeout_ms: u64,
+    ) -> Result<Value, SentinelError> {
         let iii = self.iii.clone();
         let ring = self.ring.clone();
         let function = function_id.to_string();
@@ -58,7 +69,7 @@ impl Runtime {
                 function_id: function.clone(),
                 payload,
                 action: None,
-                timeout_ms: Some(CALL_TIMEOUT_MS),
+                timeout_ms: Some(timeout_ms),
             })
             .await
             .map_err(|error| SentinelError::dependency(format!("{function}: {error}")))
@@ -67,6 +78,26 @@ impl Runtime {
     }
 }
 
+/// `judge::evaluate`, through the hub: the provider is the hub's choice.
+pub struct IiiJudge {
+    runtime: Runtime,
+}
+
+impl IiiJudge {
+    pub fn new(runtime: Runtime) -> Self {
+        Self { runtime }
+    }
+}
+
+#[async_trait]
+impl Judge for IiiJudge {
+    async fn evaluate(&self, request: Value) -> Result<Value, SentinelError> {
+        self.runtime
+            .call_within(triage::JUDGE_FUNCTION_ID, request, triage::JUDGE_WAIT_MS)
+            .await
+    }
+}
+
 /// `database::*`.
 pub struct IiiDb {
     runtime: Runtime,

sentinel/src/lib.rs

@@ -34,14 +34,15 @@ pub mod retention;
 pub mod service;
 mod status;
 pub mod store;
+pub mod triage;
 pub mod triggers;
 pub mod ui;
 
 pub use adapters::ErrorEvent;
 pub use config::{
     ArchiveConfigV1, EvidenceConfigV1, FingerprintConfigV1, IngestConfigV1, InvestigationConfigV1,
     LogSourceConfigV1, RedactionConfigV1, RepositoryConfigV1, RetentionConfigV1, SourcesConfigV1,
-    TraceSourceConfigV1, WorkerConfig,
+    TraceSourceConfigV1, TriageConfigV1, WorkerConfig,
 };
 pub use configuration::{ConfigCell, ConfigErrorCell};
 pub use contract::{
@@ -52,15 +53,16 @@ pub use contract::{
     GroupChangeReasonV1, GroupChangedConfigV1, GroupChangedEventV1, GroupChangedOpV1,
     GroupCountsV1, GroupGetRequestV1, GroupGetResponseV1, GroupHistoryRequestV1,
     GroupHistoryResponseV1, GroupStateResponseV1, GroupStatusV1, GroupSummaryV1, GroupTransitionV1,
-    GroupsListRequestV1, GroupsListResponseV1, IgnoreBaselineV1, IgnoreRequestV1, IgnoreRuleV1,
-    IngestStatusV1, InvestigateRequestV1, InvestigateResponseV1, InvestigationCancelRequestV1,
-    InvestigationChangedEventV1, InvestigationChangedOpV1, InvestigationCountsV1,
-    InvestigationGetRequestV1, InvestigationGetResponseV1, InvestigationModeV1,
-    InvestigationStatusV1, InvestigationSummaryV1, InvestigationsListRequestV1,
-    InvestigationsListResponseV1, LogsListRequestV1, LogsListResponseV1, OccurrenceSummaryV1,
-    OccurrencesListRequestV1, OccurrencesListResponseV1, ProposedFixV1, RepositoryStatusV1,
-    ResolveRequestV1, RiskV1, RootCauseV1, SourcesStatusV1, StatusRequestV1, StatusResponseV1,
-    TraceGetRequestV1, TraceGetResponseV1, TraceStoreStateV1, TurnCompletedEventV1,
+    GroupTriageV1, GroupsListRequestV1, GroupsListResponseV1, IgnoreBaselineV1, IgnoreRequestV1,
+    IgnoreRuleV1, IngestStatusV1, InvestigateRequestV1, InvestigateResponseV1,
+    InvestigationCancelRequestV1, InvestigationChangedEventV1, InvestigationChangedOpV1,
+    InvestigationCountsV1, InvestigationGetRequestV1, InvestigationGetResponseV1,
+    InvestigationModeV1, InvestigationStatusV1, InvestigationSummaryV1,
+    InvestigationsListRequestV1, InvestigationsListResponseV1, LogsListRequestV1,
+    LogsListResponseV1, OccurrenceSummaryV1, OccurrencesListRequestV1, OccurrencesListResponseV1,
+    ProposedFixV1, RelevanceV1, RepositoryStatusV1, ResolveRequestV1, RiskV1, RootCauseV1,
+    SourcesStatusV1, StatusRequestV1, StatusResponseV1, TraceGetRequestV1, TraceGetResponseV1,
+    TraceStoreStateV1, TriageKindV1, TriageSourceV1, TurnCompletedEventV1,
 };
 pub use error::SentinelError;
 pub use events::Emitter;