Skip to content

Commit e1a818c

Browse files
authored
Add Claude Code harness pilot
Adds Claude Code harness support, event-derived runtime model identity, and read-time result backfill cleanup. Validated with python3 -m unittest discover.
1 parent 65f0ff9 commit e1a818c

19 files changed

Lines changed: 1930 additions & 214 deletions

‎README.md‎

Lines changed: 34 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ The first scaffold supports:
3232
Markdown/JSON trial artifacts.
3333
- Outcome evidence including changed files and line additions/deletions.
3434
- Codex token usage capture when `codex-events.jsonl` exposes usage metadata.
35-
- Codex CLI and manual agent adapters.
35+
- Codex CLI, Claude Code, and manual agent adapters.
3636
- Multi-trial execution, concurrent trial jobs, and pass@k/pass^k summaries.
3737
- Human review labels, trial-validity metadata, and excluded-trial summaries.
3838
- Standard-library unit tests.
@@ -120,12 +120,38 @@ python3 -m agentlab run \
120120
The Codex adapter stores `codex-events.jsonl`, `codex-last-message.md`,
121121
`transcript.md`, `diff.patch`, `report.md`, and `result.json` in the run
122122
directory. Reports and result metadata include changed-file counts plus line
123-
additions/deletions from the captured patch. By default it resolves `codex` from
124-
`PATH`. If the CLI is installed outside `PATH`, fix the shell environment or use
125-
`--codex-command
126-
/path/to/codex` for that run. While the agent process is running, the terminal
127-
shows a small progress bar such as `waiting for agent response`; agent launch
128-
errors are also printed to stderr instead of only appearing in the transcript.
123+
additions/deletions from the captured patch. When JSON events expose the actual
124+
model used, `result.json` and reports derive `model_name` and `model_source`
125+
from those events; an explicit CLI `--model` is retained as the requested model
126+
rather than treated as authoritative runtime identity. By default it resolves
127+
`codex` from `PATH`. If the CLI is installed outside `PATH`, fix the shell
128+
environment or use `--codex-command /path/to/codex` for that run. While the
129+
agent process is running, the terminal shows a small progress bar such as
130+
`waiting for agent response`; agent launch errors are also printed to stderr
131+
instead of only appearing in the transcript.
132+
133+
Run a task through Claude Code:
134+
135+
```bash
136+
python3 -m agentlab doctor --agent claude
137+
python3 -m agentlab run \
138+
--agent claude \
139+
--task tasks/starter/2048-advanced-snake-params-001 \
140+
--trials 1 \
141+
--jobs 1
142+
```
143+
144+
The Claude Code adapter invokes `claude -p` in print mode, writes
145+
`claude-events.jsonl`, `claude-final-message.md`, `transcript.md`,
146+
`diff.patch`, `report.md`, and `result.json`, and records the CLI version,
147+
auth preflight status, selected model, permission mode, output format, max
148+
turns, and allowed/disallowed tool rules when configured. By default it uses
149+
`--permission-mode acceptEdits`, `--output-format stream-json`, `--verbose`, and
150+
`--no-session-persistence` for isolated trials. Like Codex, actual model
151+
identity is derived from Claude Code events when present. Use `--claude-command`
152+
when the executable is outside `PATH`, `--claude-model` to request a model, and
153+
repeated `--claude-allowed-tool` / `--claude-disallowed-tool` flags to tune the
154+
tool surface for a smoke-tested task.
129155

130156
Reference verification uses the same report/result shape, marked with
131157
`trial_kind: reference_verification`, and writes `reference-report.md`,
@@ -238,7 +264,7 @@ Issues rather than in local aggregate Markdown files.
238264
3. Add a manual adapter that lets a human edit the checkout for positive-control trials.
239265
4. Capture diffs and command results.
240266
5. Generate a Markdown report.
241-
6. Add Cursor SDK, Claude Code, and Codex CLI adapters.
267+
6. Add Cursor SDK and additional agent adapters.
242268

243269
See [CONTEXT.md](CONTEXT.md) for project vocabulary,
244270
[docs/design.md](docs/design.md) for architecture notes,

‎agentlab/__main__.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
from agentlab.cli import main
2+
import sys
23

34

45
if __name__ == "__main__":
5-
main()
6+
sys.exit(main())

‎agentlab/agents/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
from agentlab.agents.base import AgentAdapter, AgentRun
2+
from agentlab.agents.claude_code import ClaudeCodeAdapter, ClaudeCodeConfig
23
from agentlab.agents.codex_cli import CodexCliAdapter, CodexCliConfig
34
from agentlab.agents.manual import ManualAgentAdapter
45

56
__all__ = [
67
"AgentAdapter",
78
"AgentRun",
9+
"ClaudeCodeAdapter",
10+
"ClaudeCodeConfig",
811
"CodexCliAdapter",
912
"CodexCliConfig",
1013
"ManualAgentAdapter",

0 commit comments

Comments
 (0)