[claude-code-user-docs-review] Claude Code User Documentation Review - 2026-05-10 #31340

2026-05-10T12:48:05Z

github-actions[bot]
Bot May 10, 2026

Executive Summary

A developer who uses Claude Code (not Copilot) can adopt gh-aw today — the supporting reference documentation (engines, auth, tools) covers Claude well. However, the onboarding path is unmistakably Copilot-first: the Quick Start, the gh aw init flow, the introductory copy, and many sample workflows all default to Copilot, and several powerful features are Copilot-only without clearly flagged Claude alternatives. The friction is real but recoverable; no feature is blocked outright.

Key Finding: Claude Code users can succeed, but they will spend extra time stitching together info from engines.md and auth.mdx because the Quick Start, gh aw init description, and creating-workflows.mdx assume Copilot.

Persona Context

I reviewed the documentation as a developer who:

Uses GitHub for version control
Uses Claude Code as primary AI assistant
Does NOT use GitHub Copilot
Does NOT use the Copilot CLI
Does NOT have a Copilot subscription

Question 1: Onboarding Experience

Can a Claude Code user understand and get started with gh-aw?

Yes, with caveats. The Quick Start (docs/src/content/docs/setup/quick-start.mdx) does list all four AI providers in its Prerequisites and the add-wizard step explicitly mentions "Choose between Copilot, Claude, Codex, or Gemini." That is a strong start.

However, immediately after the wizard step, the prominent > [!NOTE] callout (lines 75–80) only explains how to set up COPILOT_GITHUB_TOKEN. A Claude user who picks Claude in the wizard now has to navigate away to /gh-aw/reference/auth/#anthropic_api_key to find equivalent guidance. The Copilot setup gets four numbered steps inline; Claude setup gets a hyperlink.

Specific Issues Found:

docs/src/content/docs/setup/quick-start.mdx:75-80 — The NOTE block is exclusively about COPILOT_GITHUB_TOKEN. There is no parallel callout for ANTHROPIC_API_KEY.
docs/src/content/docs/setup/creating-workflows.mdx:104 — "Running gh aw init is required to enable the authoring experience in the GitHub code agent." The described authoring experience is then entirely Copilot-based (/agent agentic-workflows in Copilot Chat, dispatcher agent in .github/agents/). This phrasing implies Claude Code users are missing something fundamental — but in practice, they author workflows from Claude Code itself, so init's Copilot-specific outputs are ornamental for them. This needs clarification.
README.md:34 and index.mdx:30 — Both lead with Copilot as the default and frame Claude/Codex as alternatives. The hero copy on index.mdx reads "Use GitHub Copilot, Claude by Anthropic or OpenAI Codex" — not bad ordering, but the supporting tutorials don't mirror this neutrality.

Recommended Fixes:

Add a parallel > [!NOTE] block in quick-start.mdx for ANTHROPIC_API_KEY setup with the same level of detail (4 numbered steps), or use tabs to switch between engines.
Reword the gh aw init section in creating-workflows.mdx to explicitly say which features are Copilot-specific (the dispatcher agent, /agent agentic-workflows) and which apply to all engines (.gitattributes, VS Code settings).
Add a short paragraph in the Quick Start: "Using Claude instead? Substitute --engine claude and ANTHROPIC_API_KEY everywhere this guide says Copilot."

Question 2: Inaccessible Features for Non-Copilot Users

What features or steps don't work without Copilot?

Most of gh-aw is engine-agnostic, but a small set of features really do require Copilot. The documentation surfaces this in the engine feature comparison table (engines.md:33-44), which is good — but it is buried in the reference and not flagged in the Quick Start.

Features That Require Copilot (genuinely Copilot-only):

engine.agent — custom agent file in .github/agents/ (Copilot only).
engine.harness — custom Node.js harness wrapper (Copilot only).
max-continuations — autopilot mode with multiple consecutive runs (Copilot only).
gh aw init's dispatcher-agent output — registers with Copilot Chat for in-browser authoring on github.com.
The web-based authoring described in creating-workflows.mdx ("GitHub Web Interface" tab) requires Copilot.
BYOK mode via COPILOT_PROVIDER_* env vars (still requires COPILOT_GITHUB_TOKEN).

Features That Require COPILOT_GITHUB_TOKEN even though they sound non-Copilot:

The Crush engine — engines.md:20 lists COPILOT_GITHUB_TOKEN as the required secret.
The OpenCode engine — engines.md:21 likewise requires COPILOT_GITHUB_TOKEN.

This is genuinely surprising — a user picking Crush because they want a non-Copilot path will not expect to need a Copilot PAT. A one-line note explaining "these engines route through GitHub's Copilot inference proxy" would prevent confusion.

Features That Work Without Copilot:

engine: claude with ANTHROPIC_API_KEY — fully supported.
max-turns — Claude only (the inverse Copilot-only feature).
All safe-outputs: types (create-issue, add-comment, create-pull-request, etc.).
tools: github:, tools: bash:, tools: edit:, tools: web-fetch:, tools: playwright:, tools: cache-memory:, tools: repo-memory:, tools: qmd:, tools: agentic-workflows:, tools: cli-proxy:.
All mcp-servers: configurations.
gh aw compile, gh aw run, gh aw logs, gh aw audit, gh aw status, gh aw health, gh aw trial.
gh aw new --engine claude, gh aw secrets bootstrap --engine claude.
gh aw mcp-server, gh aw domains, gh aw checks, gh aw fix.
Custom API endpoints via ANTHROPIC_BASE_URL for self-hosted/Azure-style proxies.

Missing Documentation:

A standalone "Using Claude with gh-aw" guide. There is engines.md (a reference) and auth.mdx, but no narrative how-to walking a Claude user through their first workflow end to end.
The phrase "Copilot is the default choice for most users because it supports the broadest gh-aw feature set" (engines.md:27) reads as a recommendation rather than a description — fine to keep, but should be balanced with a positive case for choosing Claude (e.g., when max-turns matters).
No guide explains why Crush/OpenCode use COPILOT_GITHUB_TOKEN.

Question 3: Documentation Gaps and Assumptions

Where does the documentation assume Copilot usage?

Copilot-Centric Language Found In:

README.md:34 — Quick Start link goes to a Copilot-defaulted guide; no Claude-specific quick start exists.
docs/src/content/docs/setup/quick-start.mdx:75-80 — Inline auth setup is Copilot-only.
docs/src/content/docs/setup/creating-workflows.mdx:104-127 — gh aw init description is entirely Copilot-coupled (Copilot Chat, /agent agentic-workflows, dispatcher agent).
docs/src/content/docs/setup/creating-workflows.mdx:21 — "If you have access to GitHub Copilot, you can create and edit Agentic Workflows directly from the Web Interface." This is appropriately scoped, but the alternative ("otherwise, use a coding agent") doesn't get the same prominence.
docs/src/content/docs/introduction/architecture.mdx:209,227,279 — Diagrams and config examples consistently show engine: copilot and a "Copilot CLI" box in the architecture flowchart, with Claude/Codex appearing only as labels in less prominent positions.
docs/src/content/docs/index.mdx:38 — "Supported AI engines | 4 (GitHub Copilot, Claude, OpenAI Codex, custom)" — Gemini is missing from this count even though engines.md clearly supports it. The "By the Numbers" table is out of date.
The gh aw init man-page entry (cli.md:134) says "Enables MCP server integration by default" without clarifying what MCP server — it's the Copilot-Chat-on-github.com integration.

Missing Alternative Instructions:

No "Authoring workflows from Claude Code (the IDE)" section. The creating-workflows.mdx page covers GitHub Web Interface, VSCode/Claude/Codex/Copilot CLI agents, and Manual Editing, but doesn't explicitly call out "if you use Claude Code, paste this prompt and you'll get a workflow with engine: claude already set."
No guidance comparing what init does for a Claude-Code-only user — should they still run it? (Answer appears to be: yes for .gitattributes and VS Code settings, no for the dispatcher agent.) But this isn't documented.

Severity-Categorized Findings

Critical Blockers (Score: 0/10)

None identified. A motivated Claude Code user can complete the Quick Start by substituting engine: claude and ANTHROPIC_API_KEY for the Copilot equivalents. All core features (compile, run, safe-outputs, MCP servers, GitHub tools) are engine-agnostic.

Major Obstacles (Score: 4/10)

Obstacle 1: Quick Start NOTE block only documents Copilot auth setup

Impact: A Claude user who selects Claude in the wizard sees a prominent block of instructions that don't apply to them, with no parallel block for ANTHROPIC_API_KEY.

Current State: quick-start.mdx:75-80 has a four-step > [!NOTE] for COPILOT_GITHUB_TOKEN. The ANTHROPIC_API_KEY setup is one hyperlink in the wizard's step 3.

Why It's Problematic: The visual weight of the documentation suggests Copilot is the supported path; Claude is presented as an option but not equally documented inline.

Suggested Fix: Convert the inline NOTE to a <Tabs> component (already used elsewhere in the docs, e.g., creating-workflows.mdx:28) with one tab per engine, each containing inline four-step instructions.

Affected Files: docs/src/content/docs/setup/quick-start.mdx

Obstacle 2: `gh aw init` is described as required, but its primary function is Copilot integration

Impact: A Claude Code user reads "Running gh aw init is required to enable the authoring experience" and assumes they need to do this — then discovers the authoring experience is Copilot Chat on github.com.

Current State: creating-workflows.mdx:104 makes it sound mandatory. Bullet points list dispatcher agent for Copilot, MCP integration for Copilot, .gitattributes, and VS Code settings.

Why It's Problematic: Claude users will either (a) skip init and miss the .gitattributes setup, or (b) run init and end up with a .github/agents/agentic-workflows.agent.md file they don't need.

Suggested Fix: Split the bullets into "Required for all users" (.gitattributes, VS Code settings) and "Required for Copilot Chat authoring" (dispatcher agent, MCP integration). Add --no-mcp callout for users who only need the all-users bits.

Affected Files: docs/src/content/docs/setup/creating-workflows.mdx, docs/src/content/docs/setup/cli.md

Obstacle 3: Crush/OpenCode silently require COPILOT_GITHUB_TOKEN

Impact: A user evaluating non-Copilot engines may pick Crush or OpenCode (both listed as alternative engines) and then be blocked because they don't have a Copilot PAT.

Current State: engines.md:20-21 shows COPILOT_GITHUB_TOKEN in the "Required Secret" column for both, but no explanation. The auth page (auth.mdx) doesn't mention these engines at all.

Why It's Problematic: The choice of secret is non-obvious — these are independent open-source projects, and the dependency on COPILOT_GITHUB_TOKEN reveals an implementation detail (they route through GitHub's Copilot inference proxy) that should be explicit.

Suggested Fix: Add a one-line note under the engines table: "Crush and OpenCode route inference through GitHub's Copilot proxy and therefore use COPILOT_GITHUB_TOKEN. They do not call Anthropic, OpenAI, or other provider APIs directly."

Affected Files: docs/src/content/docs/reference/engines.md, docs/src/content/docs/reference/auth.mdx

Obstacle 4: No engine-neutral or Claude-first quick start guide

Impact: Claude Code users have to mentally substitute throughout the existing Quick Start.

Current State: Single Quick Start (quick-start.mdx) defaults to Copilot at every decision point. The wizard does support engine selection, but the surrounding prose does not.

Why It's Problematic: A new user wants a copy-paste-able path. Substitution is error-prone (e.g., they may miss that add-wizard --engine claude is a flag).

Suggested Fix: Either (a) refactor the existing Quick Start to be fully engine-neutral with <Tabs> for each step, or (b) add a short "Quick Start with Claude" companion page that replaces the Copilot specifics.

Affected Files: docs/src/content/docs/setup/quick-start.mdx, possibly a new quick-start-claude.mdx

Minor Confusion Points (Score: 6/10)

Stale stat in homepage: docs/src/content/docs/index.mdx:38 says 4 engines and lists Copilot, Claude, Codex, custom — Gemini is missing, and engines.md actually lists 6 (Copilot, Claude, Codex, Gemini, Crush, OpenCode).
Architecture diagram bias: docs/src/content/docs/introduction/architecture.mdx:181 calls out only "Copilot CLI" as the agent example in the AWF flowchart.
Network configuration example uses engine: copilot only: architecture.mdx:228 shows engine: copilot in the firewall example. A Claude version would help.
engine.bare table shows different mechanism per engine but no recommendation: engines.md:325-330 — readers don't know which mechanism is "more aggressive" for Claude (--bare suppresses CLAUDE.md). Useful but unfamiliar.
CLAUDE_CODE_OAUTH_TOKEN rejection: auth.mdx:121 correctly notes this is unsupported, but a user who has Claude Max/Teams will be surprised. A short "why" sentence would help.
Engine recommendation phrasing: engines.md:27 "start with Copilot and switch later" frames Claude as a deviation rather than a peer.

Engine Comparison Analysis

Available Engines

From docs/src/content/docs/reference/engines.md:11-23, gh-aw supports six engines:

engine: copilot — default; well-documented; broadest feature support.
engine: claude — well-documented in reference, lightly covered in tutorials.
engine: codex — well-documented in reference, lightly covered in tutorials.
engine: gemini — documented in reference; missing from homepage stats; no in-repo example workflows.
engine: crush (experimental) — surprising COPILOT_GITHUB_TOKEN requirement.
engine: opencode (experimental) — surprising COPILOT_GITHUB_TOKEN requirement.

Documentation Quality by Engine

Engine	Setup Docs	Examples	Auth Docs	Overall Score
Copilot	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Claude	⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐
Codex	⭐⭐⭐	⭐⭐	⭐⭐⭐⭐	⭐⭐⭐
Gemini	⭐⭐	⭐	⭐⭐⭐	⭐⭐
Crush	⭐	⭐	⭐	⭐
OpenCode	⭐	⭐	⭐	⭐

Tool Availability Analysis

Engine-Agnostic Tools: edit:, bash:, github:, playwright:, web-fetch:, cache-memory:, repo-memory:, qmd:, agentic-workflows:, cli-proxy:, all mcp-servers:, all safe-outputs:.

Engine-Specific Tools / Configuration:

tools: web-search: — handled differently per engine (engines.md:38); Codex needs explicit opt-in, others use third-party MCP server.
tools.timeout — defaults differ per engine (tools.md:155).
engine.agent — Copilot only.
engine.harness — Copilot only.
max-turns — Claude only.
max-continuations — Copilot only.

Unclear/Undocumented:

Whether tools: agentic-workflows: (the introspection MCP server) works equivalently across engines.
Whether repo-memory: and cache-memory: have engine-specific quirks.

Authentication Requirements

Current Documentation Coverage

Copilot: documented in detail (auth.mdx:76-99), prominent in Quick Start, includes troubleshooting.
Claude: documented (auth.mdx:103-125), covers ANTHROPIC_API_KEY and the explicit rejection of CLAUDE_CODE_OAUTH_TOKEN (good), covers custom endpoints via ANTHROPIC_BASE_URL.
Codex: documented (auth.mdx:129-167), covers Azure OpenAI alternative.
Gemini: documented (auth.mdx:171-185).
Crush / OpenCode: not documented in auth.mdx; their use of COPILOT_GITHUB_TOKEN is only visible from engines.md.

Missing for Claude Users

auth.mdx:103-125 is solid as a reference but assumes the user has already chosen Claude. Linking forward from a future Claude-specific Quick Start would close the loop.

Secret Names (verified from `auth.mdx`)

Copilot: COPILOT_GITHUB_TOKEN (fine-grained PAT)
Claude: ANTHROPIC_API_KEY
Codex: OPENAI_API_KEY (or CODEX_API_KEY)
Gemini: GEMINI_API_KEY
Crush, OpenCode: COPILOT_GITHUB_TOKEN

Example Workflow Analysis

Workflow Count by Engine (from `.github/workflows/*.md` in this repo)

engine: copilot         96 workflows
engine: claude          49 workflows
engine: codex           12 workflows
engine: gemini           0 workflows
no engine (-> copilot)  25 workflows
-----
Total                  218 workflows (some count twice; approximate)

Quality of Examples

Copilot Examples: Numerous and varied. Many sample workflows in this repo use engine: copilot either explicitly or by omission.

Claude Examples: Substantial (~49). Notable ones include claude-code-user-docs-review.md (this very workflow), ci-doctor.md, audit-workflows.md, aw-failure-investigator.md. The community has a good base to copy from. However, these are in .github/workflows/ of this repo — not surfaced in docs/src/content/docs/examples/, which only shows generic engine-agnostic examples.

Recommended Actions

Priority 1: Critical Documentation Fixes

Add inline Claude (and Codex/Gemini) auth instructions to Quick Start — File: docs/src/content/docs/setup/quick-start.mdx. Use a <Tabs> component, mirroring the structure already used in creating-workflows.mdx.
Clarify what gh aw init does for non-Copilot users — File: docs/src/content/docs/setup/creating-workflows.mdx and docs/src/content/docs/setup/cli.md. Split outputs into engine-agnostic and Copilot-specific.
Document the COPILOT_GITHUB_TOKEN dependency for Crush/OpenCode — File: docs/src/content/docs/reference/engines.md and docs/src/content/docs/reference/auth.mdx.

Priority 2: Major Improvements

Refresh homepage "By the Numbers" engine count — File: docs/src/content/docs/index.mdx:38. Should be 6 (or describe Crush/OpenCode as experimental).
Add a "Why choose Claude/Codex/Gemini?" subsection to engines.md — File: docs/src/content/docs/reference/engines.md. Balance the existing Copilot recommendation.
Surface in-repo Claude example workflows in the docs site — File: docs/src/content/docs/examples/. Pick 2-3 of the 49 existing Claude workflows and link to them as case studies.
Add an "Authoring workflows with Claude Code" subsection — File: docs/src/content/docs/setup/creating-workflows.mdx. Show the prompt to paste into Claude Code, mention --engine claude for gh aw new.

Priority 3: Nice-to-Have Enhancements

Add Claude-engine config example to architecture.mdx network section.
Create a side-by-side feature parity FAQ entry: "Copilot vs Claude in gh-aw — what changes?"
Add troubleshooting entry: "I picked Claude but the Quick Start NOTE talks about Copilot — what do I do?"

Positive Findings

✅ The Quick Start prerequisites list all four AI providers up front.
✅ The add-wizard flow explicitly supports engine selection.
✅ auth.mdx is well-organized and documents ANTHROPIC_API_KEY clearly, including the explicit CLAUDE_CODE_OAUTH_TOKEN rejection.
✅ engines.md has a clear feature parity table per engine.
✅ gh aw new --engine claude and gh aw secrets bootstrap --engine claude are first-class CLI options.
✅ ~49 in-repo workflows use engine: claude — strong dogfooding signal.
✅ Custom endpoints (ANTHROPIC_BASE_URL) are documented for self-hosted/proxy setups.
✅ Engine-agnostic features (safe outputs, MCP servers, GitHub tools, Playwright, cache memory, repo memory) all work the same regardless of engine.

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Answer: Yes — with friction.

Reasoning: Every feature a Claude user needs is documented somewhere, and every CLI command supports --engine claude. The reference docs (engines.md, auth.mdx) treat Claude as a peer to Copilot. However, the introductory funnel (README → Quick Start → How It Works → Creating Workflows) is Copilot-first in tone, examples, and inline auth instructions. A Claude user will succeed but will spend extra time bouncing between pages and doing mental substitution. Closing the gap is a documentation-only effort — no code changes needed.

Overall Assessment Score: 6/10

Breakdown:

Clarity for non-Copilot users: 6/10
Claude engine documentation: 7/10 (reference is strong, tutorials are weak)
Alternative approaches provided: 6/10 (mostly via reference, not in line)
Engine parity in feature support: 8/10

Next Steps

Quickest win: edit quick-start.mdx to use <Tabs> for the auth callout — that single change fixes the most visible Copilot bias.
Second-quickest win: edit the gh aw init description in creating-workflows.mdx to label Copilot-specific outputs explicitly.
Track progress in this discussion thread and re-run this review monthly to detect regressions or improvements.

Appendix: Files Reviewed

Complete List of Documentation Files Analyzed

README.md
docs/src/content/docs/index.mdx
docs/src/content/docs/introduction/overview.mdx
docs/src/content/docs/introduction/how-they-work.mdx
docs/src/content/docs/introduction/architecture.mdx
docs/src/content/docs/setup/quick-start.mdx
docs/src/content/docs/setup/creating-workflows.mdx
docs/src/content/docs/setup/cli.md
docs/src/content/docs/reference/engines.md
docs/src/content/docs/reference/auth.mdx
docs/src/content/docs/reference/tools.md
docs/src/content/docs/reference/copilot-custom-agents.md
docs/src/content/docs/examples/manual.md
Spot-checks across .github/workflows/*.md for engine distribution.

Report Generated: workflow run §25628978647
Workflow: claude-code-user-docs-review
Engine Used: claude (eating our own dog food)

Generated by Claude Code User Documentation Review · ● 17.3M · ◷

expires on May 11, 2026, 12:48 PM UTC

2026-05-11T13:29:23Z

github-actions[bot]
Bot May 11, 2026
Author

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #31504.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[claude-code-user-docs-review] Claude Code User Documentation Review - 2026-05-10 #31340

Uh oh!

{{title}}

Uh oh!

Can a Claude Code user understand and get started with gh-aw?

What features or steps don't work without Copilot?

Where does the documentation assume Copilot usage?

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

[claude-code-user-docs-review] Claude Code User Documentation Review - 2026-05-10 #31340

Uh oh!

github-actions[bot] Bot May 10, 2026

Executive Summary

Persona Context

Can a Claude Code user understand and get started with gh-aw?

What features or steps don't work without Copilot?

Where does the documentation assume Copilot usage?

Severity-Categorized Findings

Critical Blockers (Score: 0/10)

Major Obstacles (Score: 4/10)

Minor Confusion Points (Score: 6/10)

Engine Comparison Analysis

Available Engines

Documentation Quality by Engine

Tool Availability Analysis

Authentication Requirements

Current Documentation Coverage

Missing for Claude Users

Secret Names (verified from auth.mdx)

Example Workflow Analysis

Workflow Count by Engine (from .github/workflows/*.md in this repo)

Quality of Examples

Recommended Actions

Priority 1: Critical Documentation Fixes

Priority 2: Major Improvements

Priority 3: Nice-to-Have Enhancements

Positive Findings

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Overall Assessment Score: 6/10

Next Steps

Appendix: Files Reviewed

Replies: 1 comment

Uh oh!

github-actions[bot] Bot May 11, 2026 Author

github-actions[bot]
Bot May 10, 2026

Secret Names (verified from `auth.mdx`)

Workflow Count by Engine (from `.github/workflows/*.md` in this repo)

github-actions[bot]
Bot May 11, 2026
Author