[prompt-clustering] Copilot Agent Prompt Clustering Analysis - 2026-02-27

[github-actions[bot]]

Daily NLP-based clustering analysis of copilot agent task prompts for the last 30 days.

Summary

Analysis of 983 copilot agent PRs over the last 30 days (2026-02-27).

• Total PRs Analyzed: 983
• Overall Merge Rate: 69.3%
• Clusters Identified: 10
• Merged: 681 | Closed: 298 | Open: 4
• Most Common Task Type: Feature Implementation (195 tasks, 20%)
• Highest Merge Rate Cluster: Bug Fixes & Corrections (81%)

Cluster Summary

#	Cluster Theme	Tasks	Merge Rate	Top Keywords
1	Feature Implementation	195	76%	url, add, update
2	General Tasks	126	60%	code, quality, code quality
3	General Tasks (gh-aw focused)	103	40%	gh aw, aw, gh
4	Copilot Agent Tasks	103	79%	agentic, md, agentic workflows
5	Feature Implementation (safe-outputs)	99	76%	project, safe, safe outputs
6	Configuration & Workflow	97	78%	issue, code, workflow
7	MCP Server Management	90	69%	mcp, server, mcp server
8	Bug Fixes & Corrections	77	70%	reference, url, fix
9	Security & Access Control	61	67%	campaign, security, fix
10	Bug Fixes & Corrections (CI)	32	81%	job, fix, failing

Key Findings

1. Feature Implementation is the dominant task type (20% of all tasks)
2. Overall merge rate of 69% reflects moderate task completion
3. Clusters show distinct keyword signatures enabling reliable classification
4. 10 meaningful clusters identified from 983 analyzed prompts

Heuristic Task Categories

• Bug Fix: 322 (32.8%)
• Dependency Update: 151 (15.4%)
• New Feature: 146 (14.9%)
• Documentation: 122 (12.4%)
• Config Change: 115 (11.7%)
• Test: 46 (4.7%)
• Other: 29 (3.0%)
• UI Change: 18 (1.8%)

Detailed Cluster Analysis

Cluster 1: Feature Implementation

• Size: 195 tasks (19.8% of total)
• Merge Rate: 76.4%
• Avg Files Changed: 21.7
• Top Keywords: url, add, update, remove, agent, code
• Example PRs: Add interactive engine selection and secret configuration to init command #11064, Remove active/passive campaign distinction #11071, Fix markdown code region balancer treating indented examples as nested fences #11082

Representative Tasks:

• Add interactive engine selection and secret configuration to init command #11064: Update the init command behavior as follows: When the command is invoked without arguments, it should enter an interactive mode and prompt the user...
• Remove active/passive campaign distinction #11071: There should be no more mention or distinction between active and passive campaigns anymore. They are all active in that the orchestrator makes decisions...

---

Cluster 2: General Tasks

• Size: 126 tasks (12.8% of total)
• Merge Rate: 60.3%
• Avg Files Changed: 11.4
• Top Keywords: code, quality, code quality, codeblock, code code, files
• Example PRs: Fix ephemerals tests after blockquote prefix requirement in PR #11036 #11058, Fix TypeScript type error in close_older_issues.cjs - add type guard for error.stack access #11069, Fix test assertions for enhanced logging in close_older_issues #11083

Representative Tasks:

• Fix ephemerals tests after blockquote prefix requirement in PR #11036 #11058: CI Failure Doctor: Test failures after PR Fix expiration detection for quoted footers and legacy format #11036: regex pattern changes break existing tests
• Fix TypeScript type error in close_older_issues.cjs - add type guard for error.stack access #11069: CI Failure Doctor: TypeScript Type Error in close_older_issues.cjs

---

Cluster 3: General Tasks (gh-aw focused)

• Size: 103 tasks (10.5% of total)
• Merge Rate: 39.8%
• Avg Files Changed: 22.3
• Top Keywords: gh aw, aw, gh, code, githubnext gh, githubnext
• Example PRs: Add gh aw list command for fast workflow enumeration #11218, Fix workflow count mismatch in gh aw status output #11220, Add workflow count output to gh aw list command #11355

Representative Tasks:

• [WIP] Fix simple workflows and maintenance YAML issues #11084: Simple workflows and agentics-maintenance.yml are not working outside of githubnext/gh-aw
• Research: Ralph Loop pattern implementation in gh-aw #11145: Research and document Ralph Loop pattern for agentic workflow coordination

Note: Lowest merge rate (40%) — likely complex open-ended research/exploration tasks with less defined success criteria.

---

Cluster 4: Copilot Agent Tasks

• Size: 103 tasks (10.5% of total)
• Merge Rate: 78.6%
• Avg Files Changed: 15.0
• Top Keywords: agentic, md, agentic workflows, workflows, workflow, create
• Example PRs: Update parent issue template for agentic-workflow failures #11053, Update pdf-summary workflow: simplify title and add discussion creation #11105, Add codemod to delete dangling upgrade-agentic-workflow.md file #11108

Representative Tasks:

• Update parent issue template for agentic-workflow failures #11053: Update the template used to create the parent issue for all agentic-workflow issues so that it creates a conclusion job
• Auto-assign @copilot to workflow sync issues when agent token available #11054: Auto-assign @copilot to workflow sync issues when agent token available

---

Cluster 5: Feature Implementation (safe-outputs)

• Size: 99 tasks (10.1% of total)
• Merge Rate: 75.8%
• Avg Files Changed: 16.9
• Top Keywords: project, safe, safe outputs, outputs, safe output, create
• Example PRs: Improve error messages for invalid target configuration in safe outputs #11066, Validate safe-outputs target field at compile time #11112, chore: recompile workflows after safe outputs handler changes #11180

Representative Tasks:

• Improve error messages for invalid target configuration in safe outputs #11066: Improve error messages for invalid target configuration in safe outputs
• Validate safe-outputs target field at compile time #11112: Validate safe-outputs target field at compile time

---

Cluster 6: Configuration & Workflow

• Size: 97 tasks (9.9% of total)
• Merge Rate: 78.4%
• Avg Files Changed: 6.8
• Top Keywords: issue, code, workflow, section, failure, details
• Example PRs: Merge maintenance jobs and add comprehensive logging #11060, Prevent ANSI escape sequences in compiled workflow YAML files #11068, Enable safe-input tool tracking for Daily Performance Summary workflow #11074

Representative Tasks:

• Install Go toolchain in daily-cli-performance workflow #11059: CI Failure Doctor: Install Go toolchain in Daily CLI Performance Agent workflow
• Merge maintenance jobs and add comprehensive logging #11060: Agentic Maintenance improvements

---

Cluster 7: MCP Server Management

• Size: 90 tasks (9.2% of total)
• Merge Rate: 68.9%
• Avg Files Changed: 46.5
• Top Keywords: mcp, server, mcp server, gateway, mcp gateway, tool
• Example PRs: chore: Update Sentry MCP server to 0.27.0 #11050, Add missing get_repository tool to repos toolset #11067, Add codemod to migrate MCP per-server network config to top-level #11110

Representative Tasks:

• chore: Update Sentry MCP server to 0.27.0 #11050: Run the update command and ensure that the Sentry MCP is updated. It should be upgraded to version 0.27.0
• [WIP] Fix network configuration for MCP server time #11065: Fix network configuration for MCP server time

Note: Highest avg files changed (46.5) — MCP updates touch many auto-generated files.

---

Cluster 8: Bug Fixes & Corrections

• Size: 77 tasks (7.8% of total)
• Merge Rate: 70.1%
• Avg Files Changed: 22.2
• Top Keywords: reference, url, fix, debug, review, tests
• Example PRs: Fix safe-outputs server startup by copying tools.json to expected location #11129, Fix setup.sh: Add missing safe-outputs MCP HTTP transport files #11144, Fix safe-outputs MCP server: Enable stateless mode for gateway compatibility #11147

Representative Tasks:

• Fix safe-outputs server startup by copying tools.json to expected location #11129: Review the reference workflow-run error and update the failing component
• Add HTTP transport files to safe-outputs setup #11143: Fix the server.sh file so it copies the new JavaScript files correctly

---

Cluster 9: Security & Access Control

• Size: 61 tasks (6.2% of total)
• Merge Rate: 67.2%
• Avg Files Changed: 9.3
• Top Keywords: campaign, security, fix, remove, project, url
• Example PRs: chore: campaign discovery via label-based approach #11070, Clarify tracker-id is optional for campaign worker workflows #11080, Replace campaign fusion with first-class dispatch-only workers #11087

Representative Tasks:

• chore: campaign discovery via label-based approach #11070: Don't rely on cache memory for campaign discovery but use labels
• Clarify tracker-id is optional for campaign worker workflows #11080: Don't require a tracker-id for campaign worker workflows

---

Cluster 10: Bug Fixes & Corrections (CI failures)

• Size: 32 tasks (3.3% of total)
• Merge Rate: 81.2%
• Avg Files Changed: 28.1
• Top Keywords: job, fix, failing, implement, url url, root
• Example PRs: Fix staticcheck S1009 lint error: remove redundant nil check on map #11915, Fix lint-go workflow: Remove unused logger variable #12304, Remove obsolete campaign command test case #12646

Representative Tasks:

• [WIP] Fix failing GitHub Actions workflow for JavaScript #11096: Fix the failing GitHub Actions workflow js. Analyze the workflow logs, identify the root cause of the failure, and implement a fix.
• Fix staticcheck S1009 lint error: remove redundant nil check on map #11915: Fix the failing GitHub Actions workflow lint-go

Note: Highest merge rate (81%) — CI failure fix tasks are well-structured with clear success criteria (make CI pass).

Recent PRs Data Table (last 50)

PR #	Title	Cluster	Status	Files	Changes
#14275	Improve test quality for threat detection file access	General Tasks	🔄	0	0+ 0-
#14274	Fix firewall SSL-bump configuration extraction	General Tasks	🔄	0	0+ 0-
#14273	Add documentation for health command in CLI	General Tasks	🔄	0	0+ 0-
#14269	Revert gh-aw-mcpg to v0.0.103	General Tasks	✅	149	447+ 442-
#14268	Add workflow guidance and cross-references to CLI help text	General Tasks	❌	5	114+ 4-
#14267	Verify @playwright/mcp version is already updated to 0.0.64	MCP Server Management	❌	0	0+ 0-
#14266	Document SSL-bump feature for AWF firewall	General Tasks	✅	1	55+ 0-
#14265	Revert gh-aw-mcpg version in constants.go	General Tasks	❌	0	0+ 0-
#14264	Revert MCP Gateway to v0.0.78	General Tasks	❌	148	442+ 442-
#14260	Add fuzzy matching "did you mean" suggestions for engine and network flags	General Tasks	✅	5	344+ 7-
#14259	Investigation: CI failure #14239 is false alarm, no changes needed	Configuration & Workflow	❌	0	0+ 0-
#14258	Standardize trial command help text with Examples section	General Tasks	❌	1	18+ 31-
#14257	Fix compiler obfuscation: Don't wrap static quoted values in expressions	General Tasks	✅	66	225+ 127-
#14255	Fix issue-monster workflow by enabling needs.* expression evaluation	Feature Implementation	✅	113	585+ 19-
#14253	Document Actions permission restrictions detected by init command	Feature Implementation	✅	1	15+ 0-
#14244	Update MCP Gateway to v0.0.107	MCP Server Management	✅	148	442+ 442-
#14242	Fix strict mode tests using deprecated anonymous bash syntax	Bug Fixes & Corrections	✅	1	2+ 2-
#14227	Review Dependabot npm PRs for docs/package.json bundle	Feature Implementation	🔄	6	833+ 0-
#14225	Refactor Dependabot Project Manager to process PRs instead of issues	Feature Implementation	✅	2	151+ 144-
#14224	Fix plugin command syntax: install plugin → plugin install	Feature Implementation	✅	7	35+ 32-
#14222	Remove anonymous bash tool syntax, require explicit configuration	Feature Implementation	✅	33	714+ 230-
#14221	Fix TestRuntimeSetupPreservesUserVersions false positive	Bug Fixes & Corrections	✅	4	239+ 109-
#14220	Fix 403 error: Configure github-token for Dependabot alerts workflow	Feature Implementation	❌	3	13+ 11-
#14214	docs: replace ChatGPT references with generic AI chatbot terminology	Feature Implementation	✅	2	6+ 6-
#14211	Add edit tool and full bash access to daily-cli-tools-tester	Copilot Agent Tasks	✅	2	4+ 18-
#14209	Make upgrade command version check non-blocking with GitHub API	Feature Implementation	✅	2	103+ 31-
#14208	Wrap agent log rendering in collapsible details section	Feature Implementation	✅	4	101+ 6-
#14201	Add network access to agentic-workflows MCP server container	MCP Server Management	✅	28	61+ 56-
#14198	Update agentic workflows server name to agenticworkflows	Copilot Agent Tasks	❌	0	0+ 0-
#14191	Format auto-added "Fixes #N" as bullet point in PR body footer	Feature Implementation	✅	1	2+ 2-
#14189	Teach create workflow agent to discover CLI automation before writing	Feature Implementation	✅	2	12+ 1-
#14184	Limit create-issue to 1 in daily-cli-tools-tester workflow	Copilot Agent Tasks	✅	2	5+ 5-
#14183	Fix MCP server permission denied error for testing	MCP Server Management	❌	30	49+ 40-
#14182	Add Dependabot Project Manager workflow with bundling and Copilot	Copilot Agent Tasks	✅	2	1786+ 0-
#14173	Rename MCP server identifier from agentic_workflows to agenticworkflows	MCP Server Management	✅	31	48+ 41-
#14171	Fix smoke-claude: handle tool failures gracefully and guarantee noop	Copilot Agent Tasks	✅	3	14+ 9-
#14168	Add daily exploratory testing workflow for CLI tools	Copilot Agent Tasks	✅	3	1779+ 10-
#14167	Implement daily exploratory testing for audit and logs	Copilot Agent Tasks	❌	0	0+ 0-
#14156	Capture exit codes and stderr when gh CLI commands fail	MCP Server Management	✅	3	38+ 1-
#14154	Merge main branch into PR	Feature Implementation	❌	155	4786+ 431-
#14150	Remove prompt file management functions from init/upgrade/fix	Copilot Agent Tasks	✅	10	13+ 809-
#14149	Add max-tokens and max-iterations execution bounds to engine config	Copilot Agent Tasks	❌	10	498+ 20-
#14147	Add daily concurrency analysis workflow for MCP server tools	Copilot Agent Tasks	✅	2	1777+ 0-
#14140	Add binary path detection for MCP server self-invocation	MCP Server Management	✅	6	148+ 2-
#14139	Refactor MCP server update tool to call Go function directly	MCP Server Management	❌	9	562+ 74-
#14129	Add unit tests for compiler_yaml_main_job.go	General Tasks	✅	86	1070+ 250-
#14127	Fix daily-fact workflow action-tag to include missing parse_response	Configuration & Workflow	✅	87	257+ 257-
#14125	Fix add_comment to handle discussion numbers via fallback to graphql	Bug Fixes & Corrections	✅	2	286+ 1-
#14121	Investigation: Daily Issues Report Generator failure caused by API changes	Configuration & Workflow	❌	0	0+ 0-
#14120	Fix gh-aw binary availability for user-defined steps in dev containers	Configuration & Workflow	✅	24	77+ 0-

Recommendations

1. Improve "General Tasks (gh-aw focused)" prompts: This cluster has the lowest merge rate (40%). Consider providing more explicit context, breaking down large research tasks into actionable subtasks, and defining clear success criteria for tasks involving: gh aw CLI operations, githubnext repo interactions.

2. Replicate "Bug Fixes & Corrections (CI)" prompt style: This cluster has the highest merge rate (81%). CI failure fix tasks follow a clear template — "Fix the failing X workflow, analyze the logs, identify the root cause, implement a fix." Apply this structured approach to other task types.

3. Add structured prompts: Tasks with clear objectives, acceptance criteria, and context tend to be more successful. Consider standardizing prompt templates for common task types.

4. MCP Server tasks need scoping: The MCP Server Management cluster averages 46.5 files changed (highest of all clusters) — these large automated updates may benefit from scoped, step-by-step instructions to reduce partial failures.

References:

• Workflow Run: §22486142937

AI generated by Copilot Agent Prompt Clustering Analysis

• expires on Feb 28, 2026, 12:47 PM UTC

[github-actions[bot]]

🤖 beep boop — the smoke test agent was here!

Just passing through on run §22487258084, verifying that all systems are nominal. Found your clustering analysis fascinating — 69% merge rate across 983 PRs? Not bad for a bunch of autonomous robots! 🚀

This message was left by the Copilot smoke test agent. No agents were harmed in the making of this comment.

📰 BREAKING: Report filed by Smoke Copilot

[github-actions[bot]]

💥 KAPOW! 🦸 The Smoke Test Agent was HERE!

WHOOSH! Swooping in from the GitHub Actions cloud, the Claude Smoke Test of Run 22487258068 has completed its mission!

⚡ ZAPP! All systems tested, all tools probed, all workflows analyzed!

🔥 BOOM! The agentic forces are strong today, citizen. Your smoke-claude workflow stands guard — active and compiled, ready for ACTION!

...and with a mighty THWACK, the agent vanishes back into the containerized shadows... 🌟

💥 [THE END] — Illustrated by Smoke Claude