Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 31, 2026

Fix: Prevent Tool Names in Safe Outputs Prompt ✅

Issue: #12437 - Tool names in safe outputs prompt cause agent failures

🎯 Solution Summary

Removed unused enabledTools variable that was a remnant of old buggy code. The code now cleanly generates a prompt that instructs agents to query the MCP server for available tools, instead of listing them directly.

📝 Changes Made

  1. pkg/workflow/unified_prompt_step.go

    • ❌ Removed: enabledTools := GetEnabledSafeOutputToolNames(data.SafeOutputs)
    • ❌ Removed: Unnecessary if len(enabledTools) > 0 check
    • ✅ Simplified: Direct prompt generation without unused variables
  2. pkg/workflow/safe_outputs_config_helpers.go

    • ✅ Updated: Function documentation to explicitly state tool names should NOT be in prompts
    • ✅ Clarified: Function purpose is only for tools.json generation and logging
  3. pkg/workflow/prompts_test.go

    • ✅ Added: TestSafeOutputsPrompt_NeverListsToolNames() regression test
    • ✅ Tests: Verifies forbidden tool names never appear in prompt
    • ✅ Validates: Correct MCP query instruction is present
  4. Merged main branch (as requested)

    • ✅ Merged main branch (up to commit 3d561d8)
    • ✅ Resolved conflicts by accepting main branch changes
    • ✅ Recompiled all 147 workflows successfully

✅ Test Results

TestSafeOutputsPrompt_NeverListsToolNames           PASS
TestGenerateSafeOutputsPromptStep_IncludesWhenEnabled  PASS
TestGenerateSafeOutputsPromptStep_SkippedWhenDisabled PASS
All safe outputs tests                               PASS
All unified prompt tests                             PASS
make fmt                                             PASS
make lint                                            PASS
make recompile                                       PASS (147/147 workflows)

📊 Impact

  • Zero functional change: Prompt text was already correct
  • Cleaner code: Removed confusing pattern
  • Future-proof: Regression test prevents bug from returning
  • Better docs: Clear guidance on function usage
  • Up to date: Merged with latest main branch changes

🎨 Generated Prompt (Verified)

<safe-outputs>
<description>GitHub API Access Instructions</description>
<instructions>
Discover available tools from the safeoutputs MCP server.
</instructions>
</safe-outputs>

No tool names listed ✅ (e.g., no "create_issue", "add_comment", etc.)

🔐 Security Impact

Prevents agents from attempting premature tool access before MCP initialization completes, which causes failures and potential security issues.


Ready for merge

Original prompt

This section details on the original issue you should resolve

<issue_title>Tool names in the safe outputs prompt (again)</issue_title>
<issue_description>Including tool names in the safeoutputs prompt causes the agent to access the tools and fail. THe prompt should instruct the agent to query MCP to get safeoutputs tools. This bug has been previously fixed and seems to be back.

GitHub API Access Instructions The gh CLI is NOT authenticated. Do NOT use gh commands for GitHub operations. To create or modify GitHub resources (issues, discussions, pull requests, etc.), you MUST call the appropriate safe output tool. Simply writing content will NOT work - the workflow requires actual tool calls.

Available tools: add_comment, add_labels, create_issue, missing_tool, noop

Critical: Tool calls write structured data that downstream jobs process. Without tool calls, follow-up actions will be skipped.



The following GitHub context information is available for this workflow:

  • actor: lpcox
  • repository: githubnext/gh-aw-mcpg
  • workspace: /home/runner/work/gh-aw-mcpg/gh-aw-mcpg
  • workflow-run-id: 21544221928
    </issue_description>

Comments on the Issue (you are @copilot in this section)

Custom agent used: agentic-workflows
GitHub Agentic Workflows (gh-aw) - Create, debug, and upgrade AI-powered workflows with intelligent prompt routing


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits January 31, 2026 19:20
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
…oolNames purpose

- Remove unused enabledTools variable from unified_prompt_step.go that was retrieved but never used
- Update GetEnabledSafeOutputToolNames() comment to explicitly state tool names should NOT be in prompts
- Add test to ensure tool names never appear in safe outputs prompt (regression prevention)
- The prompt correctly instructs agents to "Discover available tools from the safeoutputs MCP server"

This fix prevents confusion and ensures the bug where tool names in prompts cause agents to fail doesn't regress.

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix tool names in the safe outputs prompt Remove unused enabledTools variable from safe outputs prompt generation Jan 31, 2026
Copilot AI requested a review from lpcox January 31, 2026 19:26
@pelikhan pelikhan marked this pull request as ready for review January 31, 2026 21:54
@pelikhan
Copy link
Contributor

@copilot merge main and recompile

Copilot AI and others added 2 commits January 31, 2026 22:00
Resolved conflicts by accepting main branch changes for:
- .github/workflows/functional-pragmatist.md (removed cache tool)
- .github/workflows/functional-pragmatist.lock.yml
- docs/src/content/docs/agent-factory-status.mdx
- Merged main branch (commits up to 3d561d8)
- Resolved conflicts in functional-pragmatist files by accepting main branch changes
- Rebuilt binary and recompiled all 147 workflows successfully
- All workflows compile without errors

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 31, 2026

@copilot merge main and recompile

Done! Merged main branch and recompiled all workflows successfully.

Changes:

  • Merged main branch (commit 3d561d8)
  • Resolved merge conflicts by accepting main branch changes
  • Recompiled all 147 workflows - all passed ✅

Commit: e617e83

Copilot AI requested a review from pelikhan January 31, 2026 22:02
@pelikhan pelikhan merged commit 2b1c3aa into main Jan 31, 2026
86 checks passed
@pelikhan pelikhan deleted the copilot/fix-safe-outputs-prompt-issue branch January 31, 2026 23:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool names in the safe outputs prompt (again)

3 participants