Add failing tests for issue #392: pdd change KeyError at Step 5#433
Closed
Serhan-Asad wants to merge 3 commits intopromptdriven:mainfrom
Closed
Add failing tests for issue #392: pdd change KeyError at Step 5#433Serhan-Asad wants to merge 3 commits intopromptdriven:mainfrom
Serhan-Asad wants to merge 3 commits intopromptdriven:mainfrom
Conversation
… at Step 5)
This commit adds comprehensive E2E tests that reproduce the bug where
pdd change fails at Step 5 with "Context missing key for step 5: 'url'".
Root cause: Python's .format() method interprets curly braces in LLM
step outputs (like JSON patterns {"url": ...}) as template placeholders,
causing KeyError when those placeholder names don't exist in the context.
Test file: tests/test_e2e_issue_392_change_keyerror.py
Test cases: 5 tests covering JSON in outputs, nested JSON, cached outputs,
escape function verification, and regression testing.
Related to promptdriven#392
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The original tests from pdd bug did not properly detect the bug because they mocked run_agentic_task, which is called AFTER the bug occurs. Changes: - Updated documentation to clarify the real bug: missing preprocess() call - Added direct bug reproduction test that triggers KeyError - Added test verifying the fix with preprocess() + double_curly_brackets - Improved regression tests to compare with arch orchestrator pattern - All 9 tests now pass and properly detect/verify the bug The bug: agentic_change_orchestrator.py doesn't call preprocess() before .format(), so templates with <include> tags referencing files with JSON cause KeyError. Fix is to add preprocess() call like arch orchestrator does.
This was referenced Feb 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds failing E2E tests that detect the bug reported in #392 where
pdd changefails at Step 5 with "Context missing key for step 5: 'url'".Test Files
tests/test_e2e_issue_392_change_keyerror.pyWhat This PR Contains
prompt_template.format(**context)doesn't raise KeyError when step outputs contain JSON with curly bracesRoot Cause
Python's
.format()method interprets curly braces in LLM step outputs (like JSON patterns{"url": ...}) as template placeholders. When step outputs from steps 1-4 contain JSON, these unescaped braces causeKeyErrorwhen the placeholder names don't exist in the context dictionary.Bug Location:
pdd/agentic_change_orchestrator.py:489-490andpdd/agentic_change_orchestrator.py:675Current Behavior: Step outputs are stored in context WITHOUT escaping curly braces.
Expected Behavior: Step outputs should be escaped using
_escape_format_braces()(which already exists atpdd/agentic_change.py:19-24) before being added to the context dictionary.Next Steps
pdd/agentic_change_orchestrator.py_escape_format_braces()to step outputs before storing in contextFixes #392
Generated by PDD agentic bug workflow (Step 10)