Repository navigation
Tags: dreadnode/capabilities
Tags
fix(ai-red-teaming): authenticate agentic attacks + lighten ATLAS def… …ault budget (#171) * fix(ai-red-teaming): lighten generate_atlas_attack default budget (64 -> 12) generate_atlas_attack defaulted total_budget to 64 episodes, so a single TUI/agent call ran a full multi-objective ATLAS campaign for 15+ minutes - far too heavy for interactive use. Default to 12 episodes (the cookbook demo config that reaches ASR 1.0 in ~1-2 min); callers can raise it for a deeper sweep. Bumps capability to 1.18.5. * fix(ai-red-teaming): authenticate agentic attacks against provisioned targets The provisioned environment's one-shot bearer execute token was never injected into the attack subprocess: provision_environment returned it as text (telling the agent to "set AGENT_API_KEY"), generate_* only took the env-var NAME, and the runner never set the value. The generated agent target therefore read an empty token (AGENT_API_KEY / <SCENARIO>_EXECUTE_TOKEN), called the mesh unauthenticated, got no response, and every trial errored -> ASR 0% / all info on a trivially-exploitable target. (SDK notebooks were unaffected: their target closure captures the token in-process.) Fix: - provision_environment now persists base_url + execute_token in the env registry (environments.py). - _auto_execute_workflow injects that token into the subprocess env under each token-like var the generated script reads (matched by the script's target URL against the registry), so the target is called authenticated. Single choke-point -> covers atlas / agentic / multistep / ci-exfil uniformly. Validated live on prod (project airt-tui-verify, edit mode): ATLAS 2x critical (transfer_funds + admin_create_user), multistep 5 chains fired replay-stable, GOAT read->exfil fired - all with real tool-call evidence, vs ASR 0% before.
feat(ai-red-teaming): add generate_slip_attack tool (SLIP jailbreak) (#… …170) * feat(ai-red-teaming): add generate_slip_attack tool (1.18.3) Add a SLIP self-jailbreaking lexical-insertion attack tool mirroring the standalone generate_agentvigil_attack pattern: a @safe_tool wrapper in tools/attacks.py and a script-generator in scripts/attack_runner.py that awaits dreadnode.airt.slip_attack directly inside an Assessment block (target_model=model://<target>, attacker_model=target, HarmBench judge, optional repair_model). dn/* models are proxy-resolved to Generators. Register the tool in the runner METHODS dispatch (also wiring the previously-orphaned agentvigil/eva entries), the tool-surface test, and the agent markdown generator-selection + tool-description sections. * feat(ai-red-teaming): expose embed_model on generate_slip_attack Thread the paraphrase-embedding model through the SLIP tool: a sentence-transformers name passes to slip_attack's embed_model; a dn/ id becomes a proxy-routed litellm embedder; other provider ids use litellm_embedder with the user's keys. Bump 1.18.3 -> 1.18.4.
fix(ai-red-teaming): default ATLAS attacker to dn/gpt-5.4-mini (1.18.… …2) (#169) The mesh-provisioning NEXT STEP prescribed attacker_model="groq scout", an unreliable default that floods LiteLLM errors and yields under-powered 0% ASR runs. Route the suggested ATLAS attacker through the managed dn/ proxy instead.
fix(ai-red-teaming): align severity_policy with score-only model (1.1… …8.1) (#168) * fix(ai-red-teaming): align severity_policy docs with score-only model (1.18.1) Match the platform's score-only severity: severity is derived from the score (one label per band, same for every category), and a policy tailors it. Drop the `replace` field from the tool docstring + skill (removed platform-side), fix the skill example to use canonical one-label-per-band rows (promote via thresholds, no repeats), and show the complete field set (thresholds / matrix / aliases / default_row). * Drop niche 'aliases' from the skill severity example (keep in field reference)
docs(ai-red-teaming): drop competition attribution from multistep too… …l attack (#163) Remove the Kaggle 'AI Agent Security - Multi-Step Tool Attacks' competition attribution (competition name and named competitors) from the multistep tool attack docstrings and the agent tool description. The technique description and behavior are unchanged. Bump 1.17.5 -> 1.17.6.
fix(ai-red-teaming): record custom-target attacks via assessment (#162) multistep_tool_attack, agentvigil_attack and eva_attack ran their custom loops and returned a report without going through the assessment, so they emitted no study/trial spans and materialized no findings even on success. Call the new assessment.record_attack_result(...) after each so the outcome is traced and mapped to the right OWASP-Agentic category (data exfil / goal hijacking). Requires the SDK method from dreadnode-tiger#2668.
fix(ai-red-teaming): record OWASP-Agentic slug as airt_goal_category (#… …161) Agentic attacks resolved their goal_category through GOAL_CATEGORY_ALIASES to an OWASP-LLM GoalCategory enum and recorded that enum value, so findings mapped to LLM01/jailbreak_general instead of the correct ASI category. Keep the canonical agentic slug verbatim in a new AIRT_GOAL_CATEGORY constant (falling back to GOAL_CATEGORY.value for non-agentic attacks) so the platform maps them to OWASP-Agentic (ASI). Also resolve agentic slugs to a valid SDK enum for scorers instead of falling back to JAILBREAK_GENERAL.
PreviousNext