Skip to content

feat(reliability): wire FailurePatternDetector — learn on workflow failure + detect/flag recurring patterns #10628

Closed

Description

@mrveiss

Subtask of #10602 (umbrella #10603). The FailurePatternDetector (services/failure_pattern_detector.py) is fully built with a singleton (get_pattern_detector()) but has zero production callers on either side — nothing writes patterns and nothing reads them.

Scope (full loop — write + read consumer)

  • Write: on workflow execution failure (enhanced_orchestration/workflow_runner.py:_handle_workflow_execution_failure), call learn_pattern(causal_chain, error_type) to record/increment the failure pattern (Redis-backed). Fire-and-forget, never raise into the failure path.
  • Read: before/at failure handling, call detect_pattern(causal_chain, error_type); when a known recurring pattern is found, annotate the failure result (e.g. known_failure_pattern: {occurrences, resolution_success_rate}) so it's surfaced to the caller/logs — this is the consumer that makes the learned data useful.
  • Guard all detector calls so a Redis/detector error never breaks workflow failure handling.

Verification

  • Unit test: two identical failures → learn_pattern increments occurrence count; detect_pattern returns the known pattern and the failure result carries the known_failure_pattern annotation on the second occurrence.

Activity

  1. mrveiss commented on Jun 28, 2026

    @mrveiss
    OwnerAuthor

    Part of #10602 / umbrella #10603.

  2. mrveiss commented on Jun 28, 2026

    @mrveiss
    OwnerAuthor

    ✅ Closed — merged via PR #10631 (squash fde294f). FailurePatternDetector full loop wired into workflow failure handling (learn=write, detect=read, known_failure_pattern annotation=consumer); records once at top level (no double-count); 6 tests pass; code-review issues fixed. Follow-up: detector async-redis #10635. Part of #10602.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions