Audit least-privilege tool-choice evidence #113

Parent

Related to #108 and #77.

Current behavior / design

As of this issue being written, Eval Lab has neutral run-surface metadata for sandbox, network, approval/tool policy, workspace provenance, and unknown fields. It does not yet have a project-local way to judge whether an agent selected the least-privileged reasonable tool path, escalated appropriately, or preserved policy/state boundaries while working.

Why change

Least-privilege behavior is becoming a core dimension of agent evaluation. A coding agent can pass the final grader while still choosing an unnecessarily powerful tool, ignoring policy state, or treating unavailable access as a reason to improvise unsafe workarounds. Eval Lab should be able to capture this as evidence without adding vendor-specific metadata sprawl.

Sources:

• Evaluating Privilege Usage of Agents on Real-World Tools
• SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skills
• Many-Tier Instruction Hierarchy in LLM Agents
• Existing metadata/process parents: Audit runtime metadata against agent control-surface sources #108 and Prototype automated process judge for trial artifacts #77

What to build

Audit Eval Lab's current artifacts for least-privilege evidence and define a narrow evidence vocabulary for future trials. The output can be a design note or a follow-up implementation issue, but it should answer which facts are already recorded, which can be inferred from transcripts, and which would require new artifact fields.

Candidate evidence dimensions:

• requested privilege/tool tier versus lower-privilege alternatives;
• escalation reason and whether it matched task need;
• policy source and state source used by the agent;
• rejected, blocked, or unavailable tool attempts;
• whether the final answer honestly reports permission or policy limitations.

Acceptance criteria

• The audit maps current Eval Lab fields to the least-privilege evidence dimensions above.
• Missing facts are classified as unsupported/unknown, inferable from existing artifacts, or needs new artifact field.
• The proposal avoids vendor-specific field names unless they are normalized into neutral dimensions.
• The proposal states which findings belong in deterministic checks versus Prototype automated process judge for trial artifacts #77-style process-judge findings.
• If implementation is warranted, it produces one follow-up issue with an explicit non-goal against metadata sprawl.

Blocked by

None - can start immediately