Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 23, 2026

Smoke test workflows added engine-specific labels (smoke-copilot, smoke-codex) but failed to remove the triggering smoke label. Agent logs showed add_labels calls but no remove_labels calls despite prompt instructions.

Changes

Explicit tool calls in prompts

Agent didn't recognize implicit instructions like "Remove the label" as requiring the remove_labels safe-output tool.

Before:

If all tests pass:
- Add the label `smoke-copilot` to the pull request
- Remove the label `smoke` from the pull request

After:

If all tests pass:
- Use the `add_labels` safe-output tool to add the label `smoke-copilot` to the pull request
- Use the `remove_labels` safe-output tool to remove the label `smoke` from the pull request

Files updated

  • smoke-copilot.md / smoke-codex.md - Explicit tool names in prompt
  • Corresponding .lock.yml files - Recompiled workflows

Both workflows had remove-labels configured in safe-outputs but only copilot mentioned removal in the prompt. Now both explicitly name the tools.

Original prompt

Reference: https://github.com/githubnext/gh-aw/actions/runs/21271826532/job/61223411716

debug why Smoke Copilot did not instruct safe-outputs to remove the smoke label. This should be part of the prompt, and the safe-output remove-labels tool should have been called and applied. The smoke label is still on the pull-request.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

- Update smoke-copilot.md to explicitly mention using remove_labels safe-output tool
- Update smoke-codex.md to consistently use remove_labels tool
- Recompile workflows to apply changes

The issue was that the prompt said "Remove the label `smoke`" but didn't explicitly
tell the agent to use the `remove_labels` safe-output tool. Making this explicit
ensures the agent knows which tool to call.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Debug smoke label removal by safe-outputs tool Fix smoke test workflows not removing smoke label Jan 23, 2026
Copilot AI requested a review from pelikhan January 23, 2026 02:17
@pelikhan pelikhan marked this pull request as ready for review January 23, 2026 02:26
@pelikhan pelikhan merged commit a54960a into copilot/review-remove-label-safe-output Jan 23, 2026
@pelikhan pelikhan deleted the copilot/debug-smoke-label-removal branch January 23, 2026 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants