-
Notifications
You must be signed in to change notification settings - Fork 3.3k
chore(deployment-versioning): add migration script into repo #1318
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR introduces a data migration script (migrate-deployment-versions.ts) that facilitates the transition from storing workflow deployment state directly in the workflow table to a new versioned deployment system using a workflow_deployment_version table. The script is designed to run after manually creating the new table structure via the provided SQL commands in the PR description.
The migration script processes existing workflows in batches of 50 and attempts to populate deployment version records by first checking for existing deployedState data in the workflow table, then falling back to loading state from normalized database tables if needed. All migrated workflows are assigned version 1 and marked as active deployments. The script includes a dry-run mode for testing and comprehensive logging throughout the process.
This change is part of a larger architectural shift to support deployment versioning while maintaining backward compatibility. The script ensures that existing workflows continue to function by preserving their deployment state in the new table structure and updating the isDeployed flag to true for all processed workflows.
Confidence score: 2/5
- This PR has significant risks due to data migration complexity and potential for system-wide impact
- Score reflects concerns about unconditional deployment marking, lack of rollback mechanism, and potential SQL injection vulnerability
- Pay close attention to the migration script logic, especially the state loading fallback mechanism and the transaction handling around lines 114-138
1 file reviewed, 5 comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR adds a migration script (migrate-deployment-versions.ts) that facilitates the transition from storing workflow deployment state directly in the workflow table to a new versioned deployment system using a dedicated workflow_deployment_version table. The script is designed to run after manual table creation and processes existing workflows in batches of 100.
The migration handles two scenarios: workflows with existing deployedState (legacy format) and workflows that only exist in normalized tables (blocks, edges, subflows). For each workflow, it creates a deployment version record with version 1, storing the workflow state as JSON, and updates the original workflow record to mark it as deployed. The script includes dry-run functionality for safe testing and provides detailed logging of the migration process.
This change is part of a larger architectural shift that will enable proper deployment versioning, rollback capabilities, and better tracking of deployment history. The migration is designed to be non-destructive and idempotent, allowing multiple runs without data corruption. The business requirement to maintain continuity of scheduled workflows and webhooks during migration drives the decision to mark all processed workflows as active and deployed.
Confidence score: 2/5
- This migration script has several critical data consistency issues that could cause production problems
- Score reflects serious concerns about unconditionally marking all workflows as deployed/active and incomplete error handling
- The
migrate-deployment-versions.tsfile requires immediate attention before deployment
1 file reviewed, no comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This review covers only the changes made since the last review, not the entire PR. The most recent changes focus on adding comprehensive database migration infrastructure for a new workflow deployment versioning system. The changes include:
-
Migration Journal Update: Added entry 0088 to the migration journal (
_journal.json) to track the new migration with proper sequencing and timestamp -
Complete Schema Snapshot: Created a comprehensive database snapshot (
0088_snapshot.json) that defines the entire PostgreSQL schema including the newworkflow_deployment_versiontable alongside all existing tables -
Schema Definition Updates: Updated the Drizzle schema (
schema.ts) to include the newworkflowDeploymentVersiontable with proper TypeScript types, indexes, and foreign key relationships, while removing deprecatedparentIdandextentcolumns from theworkflowBlockstable -
SQL Migration File: Added the actual SQL migration (
0088_living_strong_guy.sql) that creates the new versioning table, establishes foreign keys and indexes, and cleans up unused columns
The deployment versioning system allows workflows to have multiple immutable deployment snapshots with version numbers, where only one version per workflow can be active at a time. This enables features like deployment history tracking, rollback capabilities, and potentially A/B testing of workflow versions. The migration also simplifies the workflow blocks structure by removing hierarchical relationship columns that are no longer needed.
Confidence score: 3/5
- This PR introduces significant database schema changes that could impact existing functionality if not handled carefully
- Score reflects the lack of data migration logic to populate the new versioning table from existing deployment data
- Pay close attention to the SQL migration file and schema changes as they modify core workflow functionality
4 files reviewed, 2 comments
* chore(deployment-versioning): add migration script into repo * add migration * remove parent id changes
…ioai#1318) * chore(deployment-versioning): add migration script into repo * add migration * remove parent id changes
Summary
Migration script to non-destructively generate rows for now deployment versions table. Should be run after psql command.
Type of Change
Testing
Manually
Checklist