Skip to content

PDD-CLI Bug: Uses Fragile CSS Utility Classes for Test Selectors #572

@jiaminc-cmu

Description

@jiaminc-cmu

PDD-CLI Bug: Uses Fragile CSS Utility Classes for Test Selectors

PDD-CLI generates E2E test selectors based on CSS utility classes (like Tailwind classes) that may not be present or may change with styling updates. This creates brittle tests that break when styling changes.

Why this matters: Tests fail when styling is updated, even though functionality is unchanged. Tests should target semantic structure, not presentation details.

Concrete Example

For a dashboard card component:

// PDD generated test (WRONG):
test('displays stage names', async ({ page }) => {
  await page.goto('/dashboard');
  
  // Selector based on Tailwind utility class
  const stageName = page.locator('.capitalize').first();
  await expect(stageName).toContainText('Lead');
});

But the component might not use that class:

// Card.tsx (actual implementation)
export function Card({ stage }: Props) {
  return (
    <div>
      <span className="text-sm font-medium">{stage.name}</span>
    </div>
  );
}

What went wrong: PDD assumed the stage name would have a .capitalize class, but the actual component uses different classes or no specific class for that element.

Impact: Test fails with locator('.capitalize') not found or finds wrong element.

Why PDD Makes This Mistake

PDD-CLI currently:

  • Infers selectors from expected styling
  • Uses CSS classes as fallback when semantic selectors unclear
  • Doesn't understand difference between utility classes and semantic classes

But it should:

  1. Never use utility classes for selectors
  2. Prefer semantic HTML attributes (role, aria-label)
  3. Generate data-testid for elements that need testing

How to Prevent This in PDD-CLI

What PDD should do differently:

  1. Use semantic selectors first:

    // Instead of CSS class
    page.locator('.capitalize')
    
    // Use role/label
    page.getByRole('heading', { name: /stage/i })
  2. Generate data-testid attributes:

    // In component
    <span data-testid="stage-name">{stage.name}</span>
    
    // In test
    page.getByTestId('stage-name')
  3. Avoid utility classes entirely: Never use .text-sm, .capitalize, .mt-4, etc. in selectors.

Example improvement:

Current: Need to select stage name → assume .capitalize class
       → Generate: page.locator('.capitalize')
       → Brittle, breaks with styling changes

Improved: Need to select stage name → generate data-testid in component
        → Component: <span data-testid="stage-name">{stage.name}</span>
        → Test: page.getByTestId('stage-name')
        → Resilient to styling changes

Severity

P3 - Low Priority

  • Frequency: Medium - occurs when PDD can't find semantic selectors
  • Impact: Tests break on styling changes (false negatives)
  • Detectability: High - fails with clear "locator not found" errors
  • Prevention cost: Low - data-testid generation is straightforward

Category

test-generation

Related Issues


For Contributors: Discovered in frontend/e2e/crm.spec.ts where .capitalize selector failed, fixed in commit 34a651d5.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions