Skip to content

Benchmark: add adversarial agent archetypes #2

@ferhatvonkaplan

Description

@ferhatvonkaplan

Context

The current benchmark generator (benchmarks/generate_benchmark.py) creates 5 agent archetypes: reliable, volatile, new, dormant, and fraudulent. Real-world adversarial agents are more sophisticated than the "fraudulent" archetype.

Task

Add 3 new adversarial archetypes to generate_benchmark.py:

  1. Slow Drift — Agent gradually shifts behavior over 30+ days to avoid anomaly detection. Small incremental changes in price, timing, counterparty concentration.
  2. Burst Manipulation — Agent behaves normally for weeks, then executes a rapid burst of anomalous transactions in a short window (<1 hour).
  3. Sybil Coordinator — Multiple agents that appear independent but coordinate to manipulate trust scores (e.g., trading with each other to inflate metrics).

Acceptance Criteria

  • 3 new archetype generators added to generate_benchmark.py
  • Each archetype produces realistic transaction patterns (not obviously synthetic)
  • evaluate.py correctly classifies these as anomalous (run and report accuracy)
  • At least 100 agents per archetype in test dataset
  • Update archetype documentation in benchmarks README

Skills needed

Python, statistical modeling, basic understanding of anomaly detection.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions