Skip to content

[agentic-token-optimizer] Issue Monster: cut per-run AIC by batching safe-output calls and trimming prompt #70278

Closed as not planned

Description

@github-actions

Target workflow

Issue Monster (.github/workflows/issue-monster.md). It is the highest-AIC workflow in the 7-day window (118.7 AIC, ahead of Daily Go Test Parallelizer at 96.2) and was last optimized 2026-09-14, outside the 14-day exclusion.

Analysis period

Last 7 days, 7 runs, all success. Engine is Pi, model copilot/gpt-5.4. turns is not reported for these runs, so I use agent invocations from working_set instead. Only per-run summary data was audited. Per-run MCP/tool call traces are not in the pre-downloaded data. The aggregate mcp_tool_usage covers all workflows, so I could not attribute tool counts to Issue Monster.

Cost profile

Metric Value
Total AIC 118.75
Avg AIC/run 16.96 (range 13.85–23.98)
Total raw tokens 1,253,142 (avg ~179K/run)
Avg invocations/run ~6.4 (range 5–10)
Avg rebuild factor ~4.8 (range 3.66–7.43)
Errors/warnings/friction 0 / 0 / 0
Avg action minutes ~9.3

Cost tracks invocation count. The 10-invocation run cost 23.98 AIC and 333K tokens. The 5-invocation runs cost 13.8–14.5 AIC and 113–123K tokens. Rebuild excess (re-sent context) is ~80% of input tokens, for example 168K of 207K in §39271700732. Every run makes 3 add_comment calls and 3 assign_to_agent calls. These are independent but appear to be issued across several turns.

Ranked recommendations

1. Batch the independent safe-output calls into a single response (~3–5 AIC/run)

Add this to the prompt (the Important note in step 5, or the issue-monster-token-budget skill):

After selecting issues, emit all assign_to_agent and add_comment calls together in one response. Do not call them one at a time or interleave them with reads.

Evidence: each run makes 6 safe-output calls for 3 issues, and invocations vary from 5 to 10 for identical work. The run-to-run variance (13.8 to 24.0 AIC) is mostly extra turns, each of which re-sends the context.

2. Make body-first validation stricter, so issue_read is the exception (~2–4 AIC/run)

Pre-activation already supplies 600-character body snippets for the top 8 candidates. Step 3 still permits issue_read for any "ambiguous" excerpt. Tighten it to: "Call issue_read only if a selected issue is not in the pre-fetched context. Never call it for the retry-blocked list. If an excerpt is ambiguous, skip the issue and pick the next candidate."

Evidence: the high-invocation runs (10 invocations, 333K tokens) look like extra read round-trips. I could not confirm this per run, because tool traces were not available.

3. Trim the prompt body (~1–2 AIC/run)

The prompt is ~280 lines and is re-sent on every invocation, so each cut is paid about 6 times per run.

  • Delete the "Filtering Applied" and "Scoring System" bullet lists in step 1. This is the pre-activation job's logic. The agent only needs the sorted list. Replace with one line: "The list is already filtered and sorted by priority; use it as is."
  • Collapse the "Topic Separation Examples" and the two integrity-blocked examples in step 3 into one short rule each.
  • Remove the issue-monster-report-formatting skill. This workflow produces no report, only comments and noop, so the skill is unused context.
  • Remove the Success Criteria block, which duplicates steps 2–5.

Estimated reduction: ~1.5K tokens per invocation.

4. Skip step 1a when no candidate has a task or plan label (~0.5–1 AIC/run)

Add a guard: "Only run step 1a if a selected issue has the task or plan label. Otherwise skip it." Without this, the agent may fetch parent and sibling issues unnecessarily.

5. Consider a smaller model (needs a trial; do not apply blindly)

The task is mostly extractive and classificatory: pick up to 3 topically distinct issues from a sorted list and call two tools per issue. Trial a smaller model (for example a mini-tier model) for 1–2 days and compare assignment quality using the existing issue_assigned eval. Potential saving is large but unmeasured.

Housekeeping

The single_issue_scoped eval asks whether "at most one issue was assigned per run", but the workflow assigns up to 3 (max: 3). Every run assigns 3. Fix the eval wording so it does not report false failures.

Structural optimizations

  • Setup prefix extraction: not warranted. The sections do not repeat opening tool calls.
  • Inline sub-agents: not recommended. The workflow is a short, sequential select-then-act flow and each section depends on the previous one. The existing skill: blocks are not sub-agents, and nothing is independent enough to justify an agent call.

Estimated savings

Recommendations 1–4 combined: ~6–10 AIC/run (~35–55% of the 16.96 average). The conservative target is ~8 AIC/run, bringing the average to roughly the 13.8–14.5 AIC floor of the best observed runs. This is based on the best-run floor, not a measurement.

Caveats

Per-run evidence
Run AIC Tokens Invocations Rebuild factor
§39271700732 18.36 209,739 7 5.32
39268326924 15.15 150,108 6 4.60
39263421904 14.02 113,460 5 3.72
39259923144 13.85 117,920 5 3.73
39255835283 14.54 122,961 5 3.66
§39252836306 23.98 333,098 10 7.43
§39246657132 18.84 205,856 7 5.05

References: §39252836306, §39271700732, §39246657132

Note: issue-monster.md has no source: frontmatter entry in the portion I read, so it can be edited directly. If it is provenance-managed upstream, apply these changes upstream first.

Generated by Agentic Workflow AIC Usage Optimizer · copilot · auto · 24.8 AIC · ⊞ 11K · ◷

  • expires on Oct 26, 2026, 7:01 AM UTC-08:00

Activity

  1. github-actions commented on Oct 20, 2026

    @github-actions
    ContributorAuthor

    This issue is being closed as outdated. A newer issue has been created: #70539

    View newer issue


    This action was performed automatically by the Agentic Workflow AIC Usage Optimizer workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions