-
Notifications
You must be signed in to change notification settings - Fork 53
Description
PDD-CLI Bug: Uses Fragile CSS Utility Classes for Test Selectors
PDD-CLI generates E2E test selectors based on CSS utility classes (like Tailwind classes) that may not be present or may change with styling updates. This creates brittle tests that break when styling changes.
Why this matters: Tests fail when styling is updated, even though functionality is unchanged. Tests should target semantic structure, not presentation details.
Concrete Example
For a dashboard card component:
// PDD generated test (WRONG):
test('displays stage names', async ({ page }) => {
await page.goto('/dashboard');
// Selector based on Tailwind utility class
const stageName = page.locator('.capitalize').first();
await expect(stageName).toContainText('Lead');
});But the component might not use that class:
// Card.tsx (actual implementation)
export function Card({ stage }: Props) {
return (
<div>
<span className="text-sm font-medium">{stage.name}</span>
</div>
);
}What went wrong: PDD assumed the stage name would have a .capitalize class, but the actual component uses different classes or no specific class for that element.
Impact: Test fails with locator('.capitalize') not found or finds wrong element.
Why PDD Makes This Mistake
PDD-CLI currently:
- Infers selectors from expected styling
- Uses CSS classes as fallback when semantic selectors unclear
- Doesn't understand difference between utility classes and semantic classes
But it should:
- Never use utility classes for selectors
- Prefer semantic HTML attributes (role, aria-label)
- Generate data-testid for elements that need testing
How to Prevent This in PDD-CLI
What PDD should do differently:
-
Use semantic selectors first:
// Instead of CSS class page.locator('.capitalize') // Use role/label page.getByRole('heading', { name: /stage/i })
-
Generate data-testid attributes:
// In component <span data-testid="stage-name">{stage.name}</span> // In test page.getByTestId('stage-name')
-
Avoid utility classes entirely: Never use
.text-sm,.capitalize,.mt-4, etc. in selectors.
Example improvement:
Current: Need to select stage name → assume .capitalize class
→ Generate: page.locator('.capitalize')
→ Brittle, breaks with styling changes
Improved: Need to select stage name → generate data-testid in component
→ Component: <span data-testid="stage-name">{stage.name}</span>
→ Test: page.getByTestId('stage-name')
→ Resilient to styling changes
Severity
P3 - Low Priority
- Frequency: Medium - occurs when PDD can't find semantic selectors
- Impact: Tests break on styling changes (false negatives)
- Detectability: High - fails with clear "locator not found" errors
- Prevention cost: Low - data-testid generation is straightforward
Category
test-generation
Related Issues
- Additional suggestions for pdd connect UI improvements #420 - Text matching instead of semantic selectors (alternative selector strategy)
- Fix #418: Clean up temp directories when git clone fails #421 - Assumes component library structure (similar selector assumption issue)
For Contributors: Discovered in frontend/e2e/crm.spec.ts where .capitalize selector failed, fixed in commit 34a651d5.