Add LINQ database query optimizations for large data scenarios #1399
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
LINQ Optimizations from PR #1302 - Rebased on Master ✅
This PR addresses database performance issues when working with workflows that have large amounts of related data. The implementation has been rebased onto the latest master branch.
Implementation Summary
✅ Core LINQ Changes Successfully Added:
AsSplitQuery()method to prevent Cartesian explosion in complex queriesAsSplitQuery()featurelargeDataOptimizedboolean parameter toUsePostgreSQL()methodlargeDataOptimizedboolean parameter toUseSqlServer()method❌ Excluded Changes (As Requested):
<TSelf>additions)✅ Latest Updates:
Technical Details
Database Query Optimization: The new
LargeDataOptimizedEntityFrameworkPersistenceProvideraddresses the non-optimal database query issue (issue #1169) by usingAsSplitQuery()in Entity Framework queries. This prevents the Cartesian product problem when loading workflows with many execution pointers and their related data.Usage:
options.UsePostgreSQL(connectionString, canCreateDB, canMigrateDB, largeDataOptimized: true)options.UseSqlServer(connectionString, canCreateDB, canMigrateDB, largeDataOptimized: true)Files Changed:
LargeDataOptimizedEntityFrameworkPersistenceProvider.cs(new file, 135 lines)WorkflowCore.Persistence.EntityFramework.csproj(EF version update)ServiceCollectionExtensions.cs(added optimization option)ServiceCollectionExtensions.cs(added optimization option)This implementation provides the exact LINQ optimization benefits from PR #1302 while maintaining a minimal changeset focused purely on the database query performance improvements.
Original prompt
💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.