Skip to content

feat(a2a): behavioural trust score for federated peers — continuous evaluation, instant downgrade on misbehaviour #7358

Closed

Description

@mrveiss

Problem

autobot-backend/a2a/security.py handles authentication (HMAC signing of Agent Cards) but does not model trust as a continuous variable. Every authenticated peer is treated equivalently — there's no notion that:

  • A peer that has succeeded on 100 prior tasks should be granted broader access than one that just authenticated
  • A peer that suddenly starts producing malformed/suspicious payloads should have its access throttled before a human investigates
  • Discovery information vs full memory access should be gated by accumulated reliability, not a static config

This only matters when AutoBot starts collaborating across trust boundaries (multi-tenant deployments, partner instances, public agent registries). Today's single-tenant deployments don't need it. But once federation is in scope, retrofitting trust scoring is much harder than designing it in.

Proposed change

New module autobot-backend/a2a/trust_score.py:

  1. Score formula — trust = 0.4×success_rate + 0.2×uptime + 0.2×threat_score + 0.2×integrity_score per peer, computed continuously
    • success_rate — fraction of tasks completed without retry/error in trailing window
    • uptime — fraction of expected heartbeats received in trailing window
    • threat_score — derived from PII pipeline events (frequent BLOCKs lower it), prompt-injection attempts, schema violations
    • integrity_score — derived from card-signature verification failures, replay attempts, TTL violations
  2. Trust levels — UNTRUSTED (≤0.3), LIMITED (0.3-0.6), STANDARD (0.6-0.85), TRUSTED (>0.85)
  3. Asymmetric updates — promotion requires sustained good behaviour over a window (e.g. 50 successful tasks); demotion is instant on any threat/integrity violation
  4. Per-level capability gates — UNTRUSTED peers see discovery info only; LIMITED can submit tasks but not query memory; STANDARD can query memory; TRUSTED can be the source of new agent definitions
  5. Persistence — scores in Redis (a2a:trust:{peer_id}), with periodic snapshot to SQLite for audit

Acceptance criteria

  • trust_score.py with formula implementation + unit tests for each component
  • Trust-level enum and per-level capability matrix documented
  • Score recomputed on every A2A interaction (success, failure, signature event)
  • Demotion path is instant; promotion requires window threshold
  • Audit-log entry on every trust-level change
  • Integration test: peer with 50 successes promoted to STANDARD; one PII BLOCK event demotes to LIMITED instantly
  • At least one production caller gates capabilities by trust level (process: systemic premature-closure pattern — 7+ feature trackers closed before integration (orchestration audit evidence) #6836 gate)
  • Snapshots flushed to SQLite for forensic audit

Why P3 (not higher)

  • Only relevant when cross-trust-boundary federation is in scope — single-tenant deployments don't benefit
  • Should be designed in before federation hardens, but doesn't need to ship before federation does
  • Builds on the PII-pipeline issue and the existing security-card flow

Dependencies

  • PII pipeline issue (provides threat_score events)
  • Existing A2A security.py (provides integrity_score events)

Out of scope

  • Reputation transfer between AutoBot instances (cross-federation reputation is a much harder problem)
  • Adversarial trust attack hardening (Sybil resistance, etc.) — defer until concrete attack model

Activity

  1. mrveiss commented on May 9, 2026

    @mrveiss
    OwnerAuthor

    Depends on #7355 (PII pipeline supplies the threat_score event stream).

  2. alex-pathcourse commented on May 10, 2026

    @alex-pathcourse
  3. github-actions commented on May 26, 2026

    @github-actions
    Contributor

    PR #8736 (merged to Dev_new_gui) references this issue with a close keyword.

    feat(a2a): behavioural trust score for federated peers (GH#7358)

    If this issue is fully resolved, close it manually. If work remains, no action is needed.

  4. mrveiss commented on May 27, 2026

    @mrveiss
    OwnerAuthor

    Resolved by PR#8838 (merged to Dev_new_gui). Continuous behavioural trust scoring for federated peers implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions