Skip to content

Structured missing-arg errors for agent elicitation#235

Merged
jeremy merged 16 commits intomainfrom
positional-args
Mar 10, 2026
Merged

Structured missing-arg errors for agent elicitation#235
jeremy merged 16 commits intomainfrom
positional-args

Conversation

@jeremy
Copy link
Copy Markdown
Member

@jeremy jeremy commented Mar 10, 2026

Summary

  • Bifurcates missing-arg behavior: TTY shows cmd.Help() (human UX), non-TTY/agent returns structured JSON errors naming the specific missing arg
  • Converts --url (webhooks create) and --question (checkins answer create) flags to positional args
  • Fixes Use: strings that marked required args as optional ([name]<name>)
  • Reorders arg checks before ensureAccount() so missing-arg errors aren't masked by auth errors
  • Updates SKILL.md with positional syntax and elicitation guidance

What agents see after this change:

{
  "ok": false,
  "error": "<content> required",
  "code": "usage",
  "hint": "Usage: basecamp todo <content>\nExample: basecamp todo \"Buy milk\""
}

The error field names the specific missing <arg> — an elicitation loop can directly prompt: "What content should the todo have?"

Test plan

  • make test — 2697 unit tests pass, zero failures
  • make test-e2e — 31 pre-existing failures (unchanged), zero new regressions, 14 pre-existing failures fixed
  • Surface snapshot and catalog parity tests pass
  • New helpers_test.go covers both TTY/agent modes, multi-line examples, isMachineOutput flag detection
  • E2e assertions use assert_json_value '.error' for exact arg-name matching

Summary by cubic

Return structured JSON usage errors for missing args and “no changes” in --agent/--json/piped runs, while keeping help output in interactive TTY. Convert key required inputs to positional args, tighten Use: strings, gate Cobra’s help-on-missing-args to interactive only, and update docs with explicit list subcommands and elicitation guidance.

  • New Features

    • Helpers: missingArg, noChanges, isMachineOutput produce { ok:false, code:"usage", error:"<arg> required", hint:"Usage: ...\nExample: ..." } for machine consumers; TTY shows help.
    • Root handler now shows help-on-missing-args only for interactive users; agents get structured errors (detects --agent, --json, piped stdout).
    • Applied across commands with clearer Use: strings and arg checks before auth. Examples:
      • Positional args required: campfire post <message>, message <title> [body], todo <content>, card <title> [content], templates create <name>, messagetypes create <name>, projects create <name>, people add/remove <person-id>..., comments list/show/update <id|url>, cards steps <card-id|url>, cards step update <step_id|url>, lineup create <name> <date>, schedule create <summary>, search <query>, url parse <url>, recordings list requires type.
      • Update commands use noChanges when no fields provided (e.g., cards/messages/files/projects/templates/messagetypes/schedule).
      • cards column color requires --color; cards step move requires --card; cards move requires --to; boost react errors with --on or --recording required.
  • Migration

    • Flags to positional args:
      • basecamp webhooks create "https://..." (replaces --url)
      • basecamp checkins answer create <question-id> "..." (replaces --question)
    • Non‑TTY/piped runs now exit non‑zero with {code:"usage"} for missing‑arg/no‑change; update scripts to read .error/.hint.
    • Docs updated: explicit list subcommands (todos/todolists/cards/messages/files/webhooks list), people uses --project for list/add/remove, subscriptions aren’t project‑scoped, and a new section explains JSON usage errors for agent elicitation.

Written for commit e4a459f. Summary will update on new commits.

@jeremy jeremy requested a review from a team as a code owner March 10, 2026 19:13
Copilot AI review requested due to automatic review settings March 10, 2026 19:13
@github-actions github-actions bot added commands CLI command implementations tests Tests (unit and e2e) skills Agent skills labels Mar 10, 2026
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 issues found across 28 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="internal/commands/tools.go">

<violation number="1" location="internal/commands/tools.go:220">
P2: Wrong argument name in structured error: `<tool-name>` should be `<id>`. The `Use` string is `update <id> <title>` and the argument is parsed as a numeric ID. An agent seeing this error would prompt for a tool name string instead of a numeric ID.</violation>

<violation number="2" location="internal/commands/tools.go:223">
P1: Copy-paste error: `<status>` should be `<title>`. This is a rename/update command — the second positional arg is a title, not a status. An agent would prompt "What status?" instead of "What title?".</violation>
</file>

<file name="internal/commands/people.go">

<violation number="1" location="internal/commands/people.go:377">
P2: Arg name `<person>...` in `missingArg` doesn't match `<person-id>...` in the `Use` string. The structured error will say `"<person>... required"` while the hint shows `add <person-id>...`. Use `"<person-id>..."` for consistency so agents see a single, unambiguous arg name.</violation>
</file>

<file name="internal/commands/todolistgroups.go">

<violation number="1" location="internal/commands/todolistgroups.go:93">
P2: `missingArg` is designed for positional args, but `todolist` here is the `--list` flag. The structured error will say `<todolist> required` with a usage hint of `basecamp todolistgroups list` — which doesn't mention `--list`, leaving agents unable to self-correct. The `create` subcommand in this same file correctly uses `output.ErrUsage("--list is required")` for the same parameter.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 94cb126dea

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves CLI “missing required input” handling to better support agent/non-interactive usage by returning structured usage errors (with the specific missing argument name), while keeping interactive TTY behavior focused on showing command help. It also updates several commands to use positional required arguments instead of flags and refreshes docs/tests accordingly.

Changes:

  • Introduces missingArg / noChanges helpers and updates many commands to return structured code:"usage" errors in machine/agent contexts.
  • Converts some required inputs from flags to positional args (notably webhooks create <url> and checkins answer create <question-id> <content>), and fixes several Use: strings to mark required args as required.
  • Updates SKILL docs, e2e assertions, and .surface snapshot to match the new interfaces and error behavior.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
internal/commands/helpers.go Adds missingArg, noChanges, and isMachineOutput helpers for structured usage errors.
internal/commands/helpers_test.go Adds unit tests covering agent vs interactive behavior and hint formatting.
internal/commands/webhooks.go Makes webhook URL positional; uses missingArg and noChanges for structured usage errors.
internal/commands/checkins.go Makes check-in answer creation use positional question-id and content; updates missing-arg handling.
internal/commands/tools.go Switches insufficient-arg handling to missingArg.
internal/commands/todolistgroups.go Uses missingArg for missing todolist selection.
internal/commands/{api,campfire,cards,comment,files,lineup,messages,messagetypes,people,projects,schedule,search,templates,todolists,todos,url}.go Replaces cmd.Help()/generic usage errors with structured missing-arg / no-changes behavior and/or reorders checks.
skills/basecamp/SKILL.md Updates documentation for positional required args and structured missing-arg errors for elicitation.
e2e/{errors,webhooks,templates,messagetypes}.bats Updates e2e expectations to assert structured JSON usage errors and new positional syntax.
.surface Removes flags that are no longer valid (--url, --question) from the surface snapshot.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot added the enhancement New feature or request label Mar 10, 2026
Copilot AI review requested due to automatic review settings March 10, 2026 19:27
@github-actions github-actions bot added bug Something isn't working and removed enhancement New feature or request labels Mar 10, 2026
@github-actions github-actions bot added enhancement New feature or request and removed bug Something isn't working labels Mar 10, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot added bug Something isn't working and removed enhancement New feature or request labels Mar 10, 2026
Copilot AI review requested due to automatic review settings March 10, 2026 19:58
@github-actions github-actions bot added enhancement New feature or request and removed bug Something isn't working labels Mar 10, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 38 out of 38 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot added bug Something isn't working and removed enhancement New feature or request labels Mar 10, 2026
Copilot AI review requested due to automatic review settings March 10, 2026 21:22
@github-actions github-actions bot removed the bug Something isn't working label Mar 10, 2026
@github-actions github-actions bot added the enhancement New feature or request label Mar 10, 2026
jeremy added 16 commits March 10, 2026 14:24
In TTY mode, missing required args show cmd.Help() (exit 0) preserving
David's human UX. In non-TTY/agent mode, return structured JSON errors
naming the specific missing arg with usage hints for elicitation.

Three new helpers in helpers.go:
- missingArg(cmd, "<arg>") — dispatches based on isMachineOutput
- noChanges(cmd) — same for update commands with no fields
- isMachineOutput(cmd) — detects agent/json/piped output

Nine unit tests covering both modes, multi-line examples, and flag
detection fallbacks.
Apply the bifurcated error pattern to ~60 call sites across all command
files. Three categories of changes:

A. Missing positional args: cmd.Help() → missingArg(cmd, "<argname>")
B. No-changes updates: cmd.Help() → noChanges(cmd)
C. Flag-to-positional: webhooks --url and checkins --question become
   positional args (cleaner surface, no dual syntax)

Also fixes Use: strings that marked required args as optional
([name] → <name>) and reorders arg checks before ensureAccount()
so missing-arg errors aren't masked by auth errors.
Tests now validate the exact JSON error contract:
- assert_json_value '.error' '<arg> required' for specificity
- assert_json_value '.code' 'usage' to verify error type
- assert_failure for commands that previously showed help (exit 0)
  in piped/non-TTY contexts

Also fixes 14 pre-existing test failures where assertions didn't
match the actual CLI output (wrong error messages, group commands
with no RunE expected to fail, removed flags).
- Quick Reference table uses positional syntax (not flags)
- New section on missing-arg errors for agent elicitation
- Resource examples updated throughout
- Removed stale "Invalid flag errors" section
After ensureProject sets app.Config.ProjectID, the local projectID
variable was reassigned but never used. Drop the dead assignments
to satisfy the ineffassign linter.
- tools update: <tool-name>/<status> → <id>/<title> (copy-paste error)
- people add/remove: <person>... → <person-id>... (matches Use: string)
- todolistgroups list: missingArg → ErrUsage for --list flag
…g changes

These flags were converted to positional args:
- webhooks create <url> (was --url)
- checkins answer create <question-id> (was --question)
- Remove unused runCmdWithFlags from edit_test.go (fixes lint)
- boost react: cmd.Help() → output.ErrUsage for missing --on flag
- lineup create: Use: [date] → <date>, ErrUsage → missingArg, reorder
  before ensureAccount
- Update assertions for new structured JSON error messages
  (assert_json_value instead of assert_output_contains)
- Update tests for removed flags (--content, --title, --list,
  --assignee, --column, --name, --date) to check "Unknown option"
- Update group commands (todos, cards, files) to assert_success
  with help output instead of assert_failure
- Fix files envelope test to use a subcommand that returns JSON
Bare group commands (e.g. `basecamp todos --in`) show help, not listings.
Add `list` after todos, todolists, cards, messages, files, and webhooks
in all examples that demonstrate listing behavior.
Subscriptions are keyed by recording ID, not project-scoped.
Normalize comment columns within each code block and remove
irregular cell padding in the Quick Reference table.
These commands use --project (required), not --in.
Replace generic `basecamp <type> --in` with `basecamp <type> list --in`
and note that some groups have default list behavior.
David's isMissingArgsError handler unconditionally shows help and exits 0
when cobra rejects a command for missing args. This breaks the structured
error contract for --agent/--json/piped consumers. Gate the help-and-exit
path behind isMachineConsumer() so machine consumers get structured JSON
errors via transformCobraError instead.
@jeremy jeremy merged commit b74b6eb into main Mar 10, 2026
24 checks passed
@jeremy jeremy deleted the positional-args branch March 10, 2026 21:28
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 43 out of 43 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jeremy added a commit that referenced this pull request Mar 10, 2026
- Expand isMachineConsumer to check --quiet/--ids-only/--count (parity
  with isMachineOutput and App.IsMachineOutput)
- Add angle brackets to missingArg calls in cards steps and recordings
  for consistency with all other commands
- Remove unused vaultTitle variable in files list (lint fix on main)
jeremy added a commit that referenced this pull request Mar 10, 2026
- Expand isMachineConsumer to check --quiet/--ids-only/--count (parity
  with isMachineOutput and App.IsMachineOutput)
- Add angle brackets to missingArg calls in cards steps and recordings
  for consistency with all other commands
- Remove unused vaultTitle variable in files list (lint fix on main)
jeremy added a commit that referenced this pull request Mar 10, 2026
- Expand isMachineConsumer to check --quiet/--ids-only/--count (parity
  with isMachineOutput and App.IsMachineOutput)
- Add angle brackets to missingArg calls in cards steps and recordings
  for consistency with all other commands
- Remove unused vaultTitle variable in files list (lint fix on main)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands CLI command implementations enhancement New feature or request skills Agent skills tests Tests (unit and e2e)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants