[prompt-analysis] Copilot PR Prompt Analysis - 2026-02-03

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-02-03

Summary

Analysis Period: Last 30 days (Jan 4 - Feb 3, 2026)
Total PRs: 1,000 | Merged: 684 (68.4%) | Closed: 315 (31.5%) | Open: 1 (0.1%)

Key Finding: Concise prompts succeed 32% more often

Merged PRs average 114 words while closed PRs average 167 words — suggesting brevity and focus drive success.

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
📝 Documentation	21	20	1	95% ⭐
♻️ Refactoring	10	8	2	80%
🧪 Testing	37	29	8	78%
🔄 Updates	59	46	13	78%
📦 Other	191	142	49	74%
✨ Features	154	113	41	73%
🐛 Bug Fixes	527	326	201	62% ⚠️

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 114 words (32% shorter than closed PRs)
• Most common keywords: workflow, github, update, agentic, safe
• Action verbs: fix, update, add, resolve, create

Keywords unique to merged PRs:

• agentic (118×) — suggests familiarity with gh-aw workflows
• safe (117×) — mentions safe outputs/inputs correctly
• create (99×) — constructive action orientation
• test (94×) — testing mentioned proactively

Example successful prompts:

1. PR #13457 (bug_fix) → Merged

   Reference: https://github.com/github/gh-aw/actions/runs/21619543844/job/62332544537#step:7:1

   Fix tests

   Why it worked: Direct, actionable, links to specific failure

2. PR #13456 (bug_fix) → Merged

   Confirmed test suite runs successfully after JavaScript refactoring changes.

   Verification

   Ran the test suite for create_issue.test.cjs as requested:

   Why it worked: Includes verification context, specific file mentioned

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 167 words (47% longer than merged)
• Most common keywords: issue_title, issue, resolve, campaign, code
• Issues identified: Overly verbose, campaign-related (multiple PRs), lack of context

Keywords unique to closed PRs:

• campaign (66×) — bulk/automated PR attempts often fail
• plan (47×) — planning without execution
• security (45×) — complex security changes harder to merge
• description (44×) — meta-information rather than action

Example unsuccessful prompts:

1. PR #13416 (bug_fix) → Closed

   Review the mcp-server command and the logic to resolve the location of the gh-aw binary. The project location changed from githubnext/gh-aw to github/aw

   Why it failed: Vague "review" directive without clear fix

2. PR #13380 (bug_fix) → Closed

   Use AWF --enable-chroot mode and remove unnecessary --mount and --env flags

   Why it failed: Complex security/sandbox changes require more context

Key Insights

🎯 Pattern 1: Conciseness wins — Merged PRs are 32% shorter on average (114 vs 167 words). Copilot performs better with focused, actionable prompts rather than lengthy explanations.

📉 Pattern 2: Bug fix paradox — Bug fixes are the MOST common prompt type (53% of all PRs) but have the LOWEST success rate (62%). Documentation changes have 95% success. This suggests bug fixes require more context or are inherently harder to automate.

🚫 Pattern 3: Campaign PRs struggle — The keyword "campaign" appears 66× in closed PRs but rarely in merged ones. Bulk/automated PR attempts have lower quality than targeted individual fixes.

Recommendations

Based on today's analysis:

✅ DO:

1. Be concise and specific — Aim for 100-150 words max. Include reference URLs to failures/issues.

   ✅ GOOD: "Fix failing test in dispatch_workflow.test.cjs (ref: run #12345)"
   ❌ BAD: "The test suite has been failing intermittently and we need to investigate why..."
   

2. Start with documentation/test tasks — These have 95%/78% success rates vs 62% for bug fixes. Build confidence with easier wins.

3. Use workflow-specific keywords — Prompts mentioning agentic, safe, workflow have higher merge rates. Shows domain knowledge.

⚠️ AVOID:

1. Campaign PRs — Creating multiple PRs at once reduces quality. Focus on one targeted fix at a time.

2. Vague action verbs — Avoid "review", "investigate", "plan". Use "fix", "add", "update", "refactor".

3. Over-explaining context — Copilot has repository access. Link to issues/runs instead of copying full descriptions.

No Historical Trends Available

This is the first day of data collection. Historical trend comparison will be available after 7 days of daily analysis.

---

Workflow Run: §21629671720
Analysis Script: /tmp/gh-aw/agent/analyze-prompts.js
Data: 1,000 Copilot PRs from last 30 days

AI generated by Copilot PR Prompt Pattern Analysis

• expires on Feb 10, 2026, 12:16 PM UTC