-
Notifications
You must be signed in to change notification settings - Fork 53
Description
PDD-CLI Bug: Generates Features Without Corresponding Test Coverage
PDD-CLI implements new features but doesn't generate tests for them. Features ship without validation that they work correctly.
Why this matters: Bugs aren't caught until production, technical debt accumulates, refactoring becomes risky.
Concrete Example
For a YAML template rendering feature:
# PDD generated feature (NO TESTS):
# template_renderer.py
import yaml
from jinja2 import Template
def render_yaml_template(template_path: str, data: dict) -> str:
"""Render YAML template with Jinja2."""
with open(template_path) as f:
template_content = f.read()
template = Template(template_content)
rendered = template.render(**data)
# Parse to validate YAML
yaml.safe_load(rendered)
return renderedBut no tests exist:
# tests/test_template_renderer.py
# FILE DOESN'T EXIST!What went wrong: PDD implemented template rendering but didn't generate tests to verify it handles edge cases (missing files, invalid YAML, special characters, etc.).
Impact: Feature ships untested. First user discovers bugs in production (file not found, YAML parsing errors, etc.).
Why PDD Makes This Mistake
PDD-CLI currently:
- Focuses on feature implementation
- Treats testing as separate step
- Doesn't follow TDD (Test-Driven Development)
But it should:
- Generate tests alongside feature code
- Cover happy path and edge cases
- Validate error handling
How to Prevent This in PDD-CLI
What PDD should do differently:
-
Generate tests with features (TDD approach):
# tests/test_template_renderer.py import pytest from template_renderer import render_yaml_template def test_render_yaml_template_basic(): result = render_yaml_template('templates/test.yaml', {'name': 'Alice'}) assert 'Alice' in result assert yaml.safe_load(result) # Valid YAML def test_render_yaml_template_missing_file(): with pytest.raises(FileNotFoundError): render_yaml_template('nonexistent.yaml', {}) def test_render_yaml_template_invalid_yaml(): with pytest.raises(yaml.YAMLError): render_yaml_template('templates/invalid.yaml', {})
-
Calculate test coverage: Ensure generated tests cover main paths.
-
Generate test data/fixtures: Create sample templates for testing.
Example improvement:
Current: "Add YAML template rendering"
→ Generate template_renderer.py
→ No tests
→ Bugs found in production
Improved: "Add YAML template rendering"
→ Generate template_renderer.py
→ Generate test_template_renderer.py
→ Generate test fixtures (sample templates)
→ Run tests to validate
→ Confident feature works correctly
Severity
P2 - Medium Priority
- Frequency: High - affects all new features
- Impact: Medium - technical debt, bugs in production
- Detectability: Low - absence of tests is passive
- Prevention cost: Medium - requires test generation logic
Category
incomplete-implementation
Related Issues
- Add failing tests for #430: auto-fix fingerprint skip bug #432 - Data models without extraction logic (similar incomplete pipeline)
- Add failing tests for #393: format injection at step 5.5 #435 - Incomplete metric calculations (similar partial implementation)
- All other issues - tests would have caught these bugs earlier!
For Contributors: Discovered when YAML template rendering was added without tests, manual tests added later in commit 34a651d5. Tests should be generated alongside features.