DevOps discipline for AI coding agents: shape the work, track it as a graph, and get each change judged by a fresh agent session that didn't write it.
AgentOps provides optional skills and a CLI (ao). The same SKILL.md skills work
with coding agents (Claude Code, Codex, Cursor, OpenCode, Gemini CLI, Pi and
others) and personal assistants (OpenClaw, Grok Bot). You state intent as
behavior in your domain's words. The skills carry it through one change
(Plan → Implement → Validate, an RPI) or, for bigger work, a goal made of
many RPIs tracked in
Beads, a dependency-aware issue tracker.
| When the agent… | AgentOps adds |
|---|---|
| Builds something different from what you meant | Given/When/Then examples shared by implementation and review |
| Uses three names for one concept | One domain term per concept, in intent, code and tests |
| Says “done” after a green test run | A fresh judge that didn't write the change |
| Loses the thread on work bigger than one session | A Beads graph holding intent, dependencies and verdicts |
| Runs off with a half-formed goal | An interview that settles the goal before agents go autonomous |
| Gives one model's answer to a hard call | A council: judges in fresh contexts, each with the model, effort and perspective you assign (one model family or several vendors), compare, duel (score each other's ideas) or debate to your majority, keep dissent, and can answer an interview for you; Idea Genie brainstorms options |
| Loses its plans, research and decisions when the session ends | Plans and decisions saved on the bead or issue (Plan, Interview, Navigate); research and idea reports under .agents/; council reports where you choose |
| Repeats the last session's investigation | Memory turns reviewed, disclosure-checked lessons into .context/ pages safe to commit |
Pick one method per agent: a plugin plus npx on the same agent gives you every skill twice.
Claude Code
claude plugin marketplace add boshu2/agentops
claude plugin install agentops@agentops-marketplace
claude plugin details agentops@agentops-marketplaceCheck that agentops appears in the plugin inventory. The bundle includes
skills, four agents and tool-call guards.
Codex
codex plugin marketplace add boshu2/agentops
codex plugin add agentops@agentops-marketplace
codex plugin list --jsonCheck that agentops appears in the inventory. Skills use the agentops: prefix;
custom roles and read limits have separate setup.
Everything else (Cursor, OpenCode, Gemini CLI, Pi, OpenClaw, Grok Bot)
With Node.js installed, run from your project directory, then pick your agents and skills:
npx skills@latest add boshu2/agentopsAdd -g for a user-level install. In scripts, name the agents:
npx skills@latest add boshu2/agentops -g -a cursor opencode -y
(-y without -a can install into every agent the installer knows).
Installer targets include cursor, opencode, gemini-cli, antigravity,
pi, grok (Grok Build) and openclaw. Grok Bot has no installer target; add
the same SKILL.md folders through its skill settings.
Some skills need extra tools (install guide); what
each host has been tested for is in host coverage and limits.
Start a new session so the skills load. Most skills need only your coding
agent; Validate also needs the ao CLI. Invocation names
vary by agent: this README shows Claude Code's /agentops:<skill>; Codex uses
$agentops:<skill>.
Each change is shaped, built and judged. You (or Plan) write intent as behavior (BDD), using one word per concept (DDD's ubiquitous language). For a system that calls queued work a Job, in Gherkin:
Feature: Job redelivery is idempotent
A Job is one unit of queued work. Delivering it again never repeats its side effect.
Scenario: A completed Job is delivered again
Given Job "J-42" completed and charged the customer $20
When the worker receives Job "J-42" again
Then it returns the completed result of "J-42"
And the customer has been charged $20 exactly once
Scenario: A Job that failed before charging is delivered again
Given Job "J-43" failed before charging the customer $20
When the worker receives Job "J-43" again
Then Job "J-43" completes
And the customer has been charged $20 exactly onceThe feature defines the domain term once; each scenario has concrete data, one
action and an observable result. Keep scenarios in the issue or conversation;
no .feature file is required.
| Step | Skill | What it does with the scenarios |
|---|---|---|
| Shape | plan |
Turns the request into scenarios for one small change. Skip it when intent is clear. |
| Build | implement |
Makes the change and tests both scenarios. |
| Judge | validate |
A new session that didn't write it returns PASS, FAIL or NOT_PROVEN against the same scenarios. |
| Learn | memory |
Optional: reviewed .context/ pages that later work can query. |
Enter at the step you need; an existing change goes straight to Validate. The author never approves its own work. Merging and releasing follow your repo's rules.
rpi runs Plan → Implement → Validate for one outcome
without check-ins (your agent's permission prompts still apply) and stops at
acceptance, a blocker or a spent limit. Bigger work becomes a goal (experimental;
needs Beads: brew install beads, then bd init in your repo):
- Interview. One question at a time, each with a recommended answer. You settle the outcome, its examples, domain terms, non-goals, authority and budget before agents go autonomous.
- Craft Goal. Returns
SAFE_TO_CREATEplus a prompt to paste into/goal(Claude Code or Codex),USE_RPI(small enough forrpi), orUNSAFE_GOALplus what's undecided. It creates nothing itself. - Navigate each round. Picks a few ready work
items (beads); each gets one RPI and a fresh Validate. The goal ends
ACHIEVED,NOT_ACHIEVEDorNEEDS_OPERATOR.
Beads holds the plan. Beads (bd)
keeps work as a dependency graph outside any conversation, so a goal survives
compaction and restarts. The root epic holds acceptance; each child bead is one
RPI with its question, scope, notes and verdict. bd ready lists what can start now.
One bead per worker. When the goal delegates, the orchestrator holds the graph and verdicts, and each worker starts with one bead instead of the orchestrator's transcript. Validators start fresh.
bd create "Job redelivery is idempotent" -t epic
bd create "Return the completed result on redelivery" --parent <epic-id>
bd dep add <later-id> <earlier-id> # real ordering only
bd ready --parent <epic-id> # the frontierNavigate shows bd commands; another tracker with status, dependencies and
notes works if you map them. AgentOps never builds a second work index.
Start read-only in any repo, then swap the Job example for your own change.
# First look (changes nothing)
/agentops:research how does this repo validate input? cite files and lines, change nothing
# One change
/agentops:plan make Job redelivery return the completed result without repeating the side effect
/agentops:implement
/agentops:validate # new session: paste the scenarios, the commit, and the authoring session's ID (your name for a hand-written change)
# One outcome, end to end
/agentops:rpi make Job redelivery return the completed result without repeating the side effect
# A goal
/agentops:interview make the job worker safe under redelivery, retries and crash recovery
/agentops:craft-goal # then paste its prompt into /goal
Validate an existing change
Pick a finished change whose accepted behavior is recorded in an issue or
conversation. Run the required checks, keep the candidate unchanged, and
install ao. Then open a new conversation, fill in the
references and paste:
Use the AgentOps Validate skill to judge this finished change.
Original accepted behavior: [issue link or original request text]
Candidate: [commit, branch or working tree; list every changed path]
Author context ID: [task/session ID that made the change]
Checks run: [commands and results]
I opened this new conversation for fresh review. Derive the exact subject
identity at the start and end, inspect every changed path against the original
behavior, and do not modify the candidate. Report PASS, FAIL or NOT_PROVEN with
evidence for each criterion, checked, not_checked, author and reviewer context
IDs, and freshness attestation.
PASS needs evidence for every criterion and an empty not_checked. FAIL
names failed behavior or an out-of-scope change. Missing proof, identity or
path coverage is NOT_PROVEN.
Read-only skill-loading smoke test
Paste this in an agent conversation in your project. It needs no ao CLI.
Use the AgentOps Research skill to trace how this repository validates user
input. Follow one path from the input through its checks and tests. Cite the
files and line numbers, explain one edge case, and identify missing coverage.
Name the Research skill file you loaded. Answer here without changing files.
The reported skill path catches missing or duplicate installs. Invoke Research
directly with /agentops:research in Claude Code, $agentops:research in
Codex, or / and the installed Research entry in Cursor.
All skills are optional. Load one when it answers a specific question. Full catalog: docs/SKILL-ROUTER.md.
| Group | Skills | What it covers |
|---|---|---|
| Operational loop | plan implement validate |
Shape, build and judge every change |
| Autonomous | rpi |
One outcome, end to end |
| Goals (experimental) | interview craft-goal navigate |
Shape, write and walk a goal over the bead graph |
| Coordination | orchestrate agent-native |
Fresh workers per bead, disjoint scopes, integration |
| On demand | research domain test refactor review security doc reverse-engineer |
Reached for when a specific question comes up |
| Learning | memory |
Curated .context/ pages safe to commit |
| Judgment strategies | council premortem postmortem reality-check idea-genie |
Multi-model councils (debates, idea duels, interview panels), idea brainstorms, plan challenges, postmortems and claim audits |
| Runtimes and factories | codex-exec agy-native using-gc |
Selected executors and Gas City integration |
| Skill craft | skill-builder skill-eval |
Author skills and measure whether they help |
AgentOps grew from applying DevOps experience and established engineering practice to agents. The Practice Registry records the lineage; how it works covers responsibilities.
Compare libraries, trackers and agent factories
| Project or tool | Role alongside AgentOps |
|---|---|
| Compound Engineering | A connected development workflow and reusable solution records |
| Matt Pocock's skills | Composable practices for intent, domain modeling, TDD and review |
| Beads or your existing tracker | Owns work status, dependencies and handoffs |
| Factories such as Gas City | Own agent coordination and execution through their native control plane |
Choose which workflow leads the task. Carry accepted behavior and evidence into independent judgment. Shared practices are not proof that every combination has been tested.
Most skills need only your coding agent. Validate uses ao to identify the
exact change it judges.
brew tap boshu2/agentops
brew trust --tap boshu2/agentops
brew install agentops
ao versionWith Go installed: go install github.com/boshu2/agentops/cli/cmd/ao@latest.
ao init is optional evidence setup; ao config --show inspects configuration;
ao gate check runs this repository's own gates (mainly for contributors). See the command reference and
installation guide.
Upgrading to 3.8
Version 3.8 retains existing 3.7 command and skill names. Use the
plugin update instructions
or, for npx installs, npx skills@latest update (update notes).
For Homebrew: brew update && brew upgrade agentops. Start a new session
afterward; new installs do not silently remove obsolete copies.
Upgrading from 3.6 or earlier: read the migration guide.
Version 3.7 removed commands and skill names, including learn, codebase-recon
and swarm; their current owners are memory, research and agent-native.
See the 3.8 release notes and
3.7 removals.
Skill dependencies
Skill installation does not install tool dependencies:
| Skill | Needs | Why |
|---|---|---|
rpi |
ao, conditional |
delegates exact-subject checks to Validate; only persists verdict.v2 when requested, with the fixed-dispatch adapter optional |
plan |
ao, conditional |
runs ao provenance snapshot-intent with an explicit evidence root when the intent source is not durable |
implement |
ao, conditional |
at an integration boundary whose changed paths affect bound evidence, runs ao provenance evidence-orphans |
validate |
ao |
derives exact subject identity with the helper and uses ao provenance store-verdict when persistence is requested; Python/schema checks are developer-only |
reality-check |
ao, conditional |
inspect selected goal measurements with ao goals or evidence-store facts with ao status |
using-gc |
ao |
rig prep runs ao gc prepare and ao gc check |
doc |
ao, optional |
a requested continuity handoff may use ao session handoff/rehydrate |
reverse-engineer |
python3 |
Phase 1's mechanical teardown runs scripts/reverse_engineer.py |
skill-builder |
python3, conditional |
Create mode's build.sh runs scripts/generate-skill-mesh.py; heal/check/audit modes are bash-only |
memory |
python3, conditional |
a selected toil investigation can use the repository helper scripts/toil-mining/recent_human.py on cleared Codex sources |
security |
python3, conditional |
the composable suite and offline redteam surfaces run security_suite.py when that scan type is selected |
Shared project context
Memory can read reviewed .context/ notes with ordinary
filesystem tools; no ao or Beads is needed. See this repo's context map.
Adding notes requires authorized sources and fresh review of factual support and disclosure before Git admission. Drafts and evidence stay in protected external storage. These procedures do not enforce access permissions or prove that saved notes improve later work. See Memory's storage rules.
Permissions, optional hooks, and removal
The Claude Code plugin includes PreToolUse guards for private tracker data in
commits, manual provenance-ledger edits and installed-skill overwrites. Installing
only ao does not add hooks; other paths can opt in through the native hook installer.
Read-budget guards, Codex roles and trusted Codex hooks have separate setup.
Disable Claude's plugin with /plugin disable agentops. Remove it with
claude plugin uninstall agentops@agentops-marketplace; for Codex, use
codex plugin remove agentops@agentops-marketplace; for npx installs, use
npx skills@latest remove.
Architecture and saved review evidence
AgentOps is the operations layer for agentic engineering. Its federated integration graph connects evidence while Git owns content, the tracker owns work and the coding runtime or selected factory owns execution. Your repository owns delivery.
Native execution requires zero AgentOps skills. RPI, Gas City and Agentic Coding Flywheel are optional; their completion reports do not replace independent review.
On request, Validate can save verdict.v2 with exact content, checked scope and
evidence. New proof belongs in selected, protected storage outside Git;
existing evidence is preserved.
Read the architecture, operating contract and storage rules.
Common problems and how to report one
| Symptom | What to check |
|---|---|
plugin is not recognized |
Update your agent to a version with plugin support |
| A skill is missing | Check its inventory or picker, then start a new session |
ao is not found |
Install the CLI and check PATH; Go installs usually use $(go env GOPATH)/bin |
| A skill needs another tool | Check its dependencies in the installation guide |
| An old skill name fails | Check the migration guide and stale copies |
Report a reproducible issue with your runtime version, install method, command or prompt, and observed result. Share only evidence you are authorized to disclose.
Skills guide agents; installation alone does not enforce their instructions.
A green test suite or an agreeing model can still miss a defect. Missing proof
stays NOT_PROVEN. Saving notes does not establish improved outcomes or
automatic knowledge compounding. See product evidence and limits.
CLI, reviewer model and storage questions
Do I need the CLI, an orchestrator or several agents?
No. Start with one coding agent and a skill such as Research, Test or Refactor.
Obtain fresh, author-distinct judgment when a change is ready. The Validate skill
requires ao; native independent review does not.
Must the reviewer use another model provider?
No. The default is a fresh context from the author's model family. Cross-model review is an explicit choice; independence still matters.
Must durable work live in Git?
No. Keep source in Git, handoffs in your tracker and requested proof in protected external storage, following each owner's access and retention rules.
These independent projects can extend your AgentOps setup. Get their tools and skills directly from their authors; they are not bundled with AgentOps.
- Gas City, from the organization behind Beads, provides the building blocks for a software factory. AgentOps includes Gas City guidance and an executor integration pack.
- Agentic Coding Flywheel is Jeffrey Emanuel's (Dicklesworthstone) ecosystem for planning and coordinating coding agents. Follow his site for the tools, methodology and upstream skills.
- Destructive Command Guard (
dcg) is a strong recommendation for checking agent shell commands and blocking destructive operations covered by its rules. - Coding Agent Session Search (
cass) is a strong recommendation for indexing and searching session histories across coding agents, including finding episodes worth mining for lessons. - Meta Skill (
ms) is a strong recommendation for finding, managing and building skills from context, including CASS session mining. Review generated guidance before reuse.
DCG, CASS and MS are Jeffrey Emanuel's projects. Their upstream documentation and distribution terms govern their tools and skills.
Contributions are welcome: documentation fixes, reproducible bug reports,
tests, CLI improvements and skills. Read the contribution guide;
to work on skills from a checkout, link them with
ao skills link.
Licensed under Apache-2.0.