Skip to content

Add prompt provenance, tracing, evals, and failure taxonomy #32

Description

@Konan69

Current state

Garden imported the core prompt posture and has document citations through findInDocument, issue/automation run traces, and bundled document skills. It does not have a product-wide provenance, prompt snapshot, tracing, or eval layer.

Remaining gaps

  • General citation/provenance system across tools and claims.
  • Unified clarification contract across chat, issues, automations, and external messengers.
  • File-delete and destructive-operation gating.
  • Observability/eval suites.
  • Prompt/version tracking and per-run snapshot storage.
  • Consolidated chat tool trace UI/export.
  • Failure taxonomy.
  • Confirm whether the planned schedule skill is installable or remove it from planning docs.

Acceptance criteria

  • Each run can be inspected with prompt version/snapshot, selected tool traces, and model/provider metadata.
  • Claims that cite documents or external sources have structured provenance where the tool supports it.
  • Destructive operations share a clear approval policy.
  • Eval coverage exists for prompt/tool safety regressions.

Source

docs/known-gaps/prompt-and-system-spec.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions