Skip to content

FEAT Add runtime Techniques registry to the GUI - #3047

Merged
Richard Lundeen (richlundeen) merged 19 commits into
microsoft:mainfrom
richlundeen:richlundeen-techniques-registry-ui
Oct 10, 2026
Merged

Richard Lundeen (richlundeen) merged 19 commits into
microsoft:mainfrom
richlundeen:richlundeen-techniques-registry-ui

Conversation

@richlundeen

@richlundeen Richard Lundeen (richlundeen) commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds Techniques beside Targets and Converters in the GUI Registry. Users can search and filter the active technique catalog, inspect a short factory creation call, and add named configurations of registered attack classes. This is the follow-up to #2952.

  • Uses real AttackTechniqueFactory instances and the existing AttackRegistry; there is no second attack catalog or live attack-instance registry.
  • Adds authenticated list, type-metadata, detail, and creation endpoints. The list uses cursor pagination, and the GUI follows all pages for search and filters.
  • The basic form accepts supported attack parameters, ordered converter references, and optional adversarial target/prompt settings. Advanced seeds, conversation settings, and live Python values remain available through programmatic factories and initializers. Creating or viewing a technique does not execute an attack or send prompts.
  • Refreshes compatible scenario selection metadata and estimate/summary caches when the factory pool changes. Existing scenario snapshots and active runs stay unchanged; fixed catalogs retain their own pools, and scenario-local factories take precedence.
  • Captures supplied factory arguments for the details view instead of displaying identifiers, hashes, or expanded internal settings. Component credentials are not included.
  • Rejects selector collisions, including inherited scenario-enum names, before registration. The form hides unsupported nested structured settings and uses the shared page loader with explicit errors for incomplete reference lists.

Additions are runtime-only. A restart or reinitialization removes them. This feature does not change dependencies or database schemas. Current main is merged, and its conflicts are resolved.

Screenshots

Technique list — Search and filter the active factory catalog.

Technique registry list with search and attack-type/tag filters

Technique details — Show the supplied factory creation call without identifiers or expanded internal settings.

Details for crescendo_simulated showing its factory helper call

Add a technique — Configure an existing attack with a name, tags, supported parameters, and converter references.

New technique form with PromptSendingAttack and a Base64 request converter

Tests and Documentation

Updated the backend API documentation and added backend, registry, GUI, API-client, and real-backend browser coverage. The regressions cover atomic admission, cache refresh, old snapshots, creation-call display, reference order, false/zero/empty values, nested form capabilities, pagination, and stale runtime responses.

All commands below passed after merging current main. Commands ran from the repository root, except the frontend commands, which ran from frontend:

Command Result
uv run --no-sync pytest tests\unit\registry\test_attack_technique_registry.py tests\unit\registry\test_attack_technique_construction.py tests\unit\backend\test_technique_service.py tests\unit\backend\test_scenario_service.py tests\unit\scenario\core\test_attack_technique_factory.py tests\unit\scenario\airt\test_leakage.py tests\unit\scenario\airt\test_multilingual.py tests\unit\scenario\scenarios\adaptive\test_text_adaptive.py -q Passed: 407 tests
uv run --no-sync pytest tests\unit\backend\test_scenario_run_service.py -q Passed: 154 tests
npm test -- --watch=false --runInBand TechniqueRegistry.test.tsx fetchAllPages.test.ts api.test.ts RegistryLayout.test.tsx parameterForm.test.ts ParameterField.test.tsx Passed: 142 tests in 5 matching suites
npm run build Passed
npx eslint src\components\Registry\CreateTechniqueDialog.tsx src\components\Registry\TechniqueRegistry.test.tsx src\utils\fetchAllPages.ts src\utils\fetchAllPages.test.ts Passed
uv run --no-sync pre-commit run --all-files with UV_NO_SYNC=1 All hooks passed, including full Python type checking
npm run test:e2e -- 'registry\.spec\.ts' --project seeded --grep 'creates and selects a runtime technique' --retries 0 --reporter list with dedicated seeded servers Passed: 1 real-backend browser test

The browser check verifies runtime registration and scenario selection without prompt sends. It ran with CI=true, E2E_SEEDED_MODE=true, E2E_FRONTEND_PORT=18843, and PYRIT_E2E_BACKEND_PORT=18842. Build identities were stamped before starting both servers.

JupyText was not run: no paired Python/notebook documentation changed. The full unit suite was not run; the scoped commands above cover the affected behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove TechniqueDefinition and the parallel structured-input resolver. Keep REST creation to basic attack parameters, converter references, and inline adversarial settings. Advanced seeds and conversation settings remain Python-only.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Restore the registry resolver contract to main. Limit REST input checks to the technique service and remove duplicate runtime technique documentation from the registry guide.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reject inherited scenario enum names before runtime registration, keep structured forms within shared resolver capabilities, and reuse complete-list pagination.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep both imports required by the fixed-catalog regression and current scenario run-plan tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@romanlutz Roman Lutz (romanlutz) self-assigned this Oct 9, 2026
Comment thread pyrit/registry/components/attack_technique_registry.py
Comment thread pyrit/registry/components/attack_technique_registry.py
Comment thread pyrit/backend/services/technique_service.py
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve compatible resolved selections across factory additions. Validate candidates against scenario-local pools before registration, and reuse attack-owned configuration checks for deferred construction.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread pyrit/backend/services/technique_service.py
Comment thread pyrit/registry/components/attack_registry.py Outdated
Preserve revision-aware technique catalogs with explicit dataset sources and limits.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@richlundeen
Richard Lundeen (richlundeen) added this pull request to the merge queue Oct 10, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 10, 2026
@richlundeen
Richard Lundeen (richlundeen) added this pull request to the merge queue Oct 10, 2026
Merged via the queue into microsoft:main with commit e993f0d Oct 10, 2026
55 checks passed
@richlundeen
Richard Lundeen (richlundeen) deleted the richlundeen-techniques-registry-ui branch October 10, 2026 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants