Skip to content

BUG: analytics no_converters bucket never matches real attacks #3088

Description

The no_converters bucket of the converter_type analytics dimension never matches a real attack.

The bucket is only set when the stored converter list is an empty JSON array (attack_analytics_query.py, the JsonArrayEmpty case). A real attack with no converters doesn't store an empty list, it leaves the key out, so it lands in missing instead.

Repro: run one PromptSendingAttack with no converters against a mock target, then query with AttackAnalyticsReader:

request groups: [('missing', {'undetermined': 1})]
request filter no_converters -> {}
response groups: [('missing', {'undetermined': 1})]
response filter no_converters -> {}
memory.get_attack_results_async(has_converters=False) -> 1

So filtering on "no converters" in analytics returns nothing, while the history API's has_converters=False finds the attack. Treating an absent list as no_converters for converter_type (when the attack identifier itself was recorded) would line the two up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions