Skip to content

refactor(agent): change PM Agent output from businessRequirement/functionalRequirements to goal/testcases #3665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

refactor(agent): change PM Agent output from businessRequirement/functionalRequirements to goal/testcases #3665

wants to merge 2 commits into from
+771 −546

Conversation

devin-ai-integration[bot]
Copy link
Contributor

Issue

This refactoring addresses a structural change request to modernize the PM Agent's output format and improve the integration between PM and QA agents.

Why is this change needed?

The previous architecture had PM Agent generating businessRequirement and functionalRequirements, with QA Agent maintaining separate testcases arrays. This created complexity in state management and data synchronization. The new structure has PM Agent creating test case structures (without SQL) that QA Agent then fills in, providing a cleaner data flow.

Summary of Changes

Core Schema Changes:

  • Updated analyzedRequirementsSchema from businessRequirement/functionalRequirements to goal/testcases
  • Added testCaseSchema with title, type, sql, and testResults fields
  • Removed separate testcases field from workflow annotation

Agent Updates:

  • PM Agent now creates test case structures without SQL
  • QA Agent updated to fill in SQL for existing test cases instead of creating separate arrays
  • Simplified QA agent processing logic in getUnprocessedRequirements.ts

Supporting Changes:

  • Complete rewrite of transformStateToArtifact.ts with simplified logic
  • Updated all UI formatting functions and Storybook stories
  • Updated 20+ test files to use new field names
  • Updated fixture files with new structure

Key Areas for Review

⚠️ Critical - State Management Flow: Verify that QA Agent can properly read and update test cases within the new analyzedRequirements.testcases structure

⚠️ Critical - Artifact Generation: The transformStateToArtifact.ts logic was completely rewritten and simplified - ensure it produces correct output for all scenarios

⚠️ High Risk - Test Case Processing: Confirm that test cases don't get duplicated or lost in the new flow, especially in getUnprocessedRequirements.ts which was significantly simplified

⚠️ Integration Testing: Verify that the full PM → QA agent workflow works correctly with the new embedded testcase structure

⚠️ Schema Validation: Ensure the new schema properly validates and the type system catches any mismatched usage

Testing Status

  • ✅ All lint checks pass
  • ✅ All unit tests updated and passing
  • ❌ No runtime integration testing performed - requires manual verification of agent workflow

Link to Devin run: https://app.devin.ai/sessions/bbcc9bc20c404244b5dc51b92e0cb8fd
Requested by: @MH4GF

@devin-ai-integration @MH4GF
refactor(agent): change PM Agent output from businessRequirement/func…
ac9a969 
…tionalRequirements to goal/testcases

- Update analyzedRequirements schema to use goal and testcases structure
- Add testCaseSchema with title, type, sql, and testResults fields
- Remove separate testcases field from workflow annotation
- Update PM Agent to create test case structures without SQL
- Update QA Agent to fill in SQL for existing test cases
- Update all utilities (transformStateToArtifact, convertAnalyzedRequirementsToPrompt)
- Update UI formatting in formatArguments.ts
- Update all test files and fixtures to use new structure
- Update Storybook stories with new field names

Co-Authored-By: hirotaka.miyagi@route06.co.jp <h.miyagi.cnw@gmail.com>
@devin-ai-integration Devin.ai Integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@changeset-bot changeset-bot
Copy link

changeset-bot bot commented Oct 3, 2025 •
edited
Loading

⚠️ No Changeset found

Latest commit: 0422f17

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel Vercel
Copy link

vercel bot commented Oct 3, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
liam-app Ready Ready Preview Comment Oct 3, 2025 7:45am
liam-assets Ready Ready Preview Comment Oct 3, 2025 7:45am
liam-storybook Ready Ready Preview Comment Oct 3, 2025 7:45am
2 Skipped Deployments
Project Deployment Preview Comments Updated (UTC)
liam-docs Ignored Ignored Preview Oct 3, 2025 7:45am
liam-erd-sample Skipped Skipped Oct 3, 2025 7:45am

@coderabbitai coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 3, 2025

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Comment @coderabbitai help to get the list of available commands and usage tips.

@supabase Supabase
Copy link

supabase bot commented Oct 3, 2025 •
edited
Loading

Updates to Preview Branch (devin/1759475466-refactor-pm-agent-output-structure) ↗︎

Deployments Status Updated
Database ✅ Fri, 03 Oct 2025 07:40:53 UTC
Services ✅ Fri, 03 Oct 2025 07:40:53 UTC
APIs ✅ Fri, 03 Oct 2025 07:40:53 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks Status Updated
Configurations ✅ Fri, 03 Oct 2025 07:40:54 UTC
Migrations ✅ Fri, 03 Oct 2025 07:40:54 UTC
Seeding ✅ Fri, 03 Oct 2025 07:41:08 UTC
Edge Functions ✅ Fri, 03 Oct 2025 07:41:11 UTC

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

@devin-ai-integration @MH4GF
fix(agent): correct requirementId filtering to use position-based ind…
0422f17 
…exing

The filtering logic was incorrectly trying to match test case titles
against requirementIds. Updated to use 1-indexed position-based
filtering across all test cases as expected by the tests.

Also refactored to reduce function complexity from 16 to 15 by
extracting buildPositionMap helper function, and fixed TypeScript
error with non-null assertion.

Co-Authored-By: hirotaka.miyagi@route06.co.jp <h.miyagi.cnw@gmail.com>
@vercel vercel bot temporarily deployed to Preview – liam-erd-sample October 3, 2025 07:40 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@MH4GF MH4GF

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@MH4GF