Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Sep 29, 2025

LINQ Optimizations from PR #1302 - Rebased on Master ✅

This PR addresses database performance issues when working with workflows that have large amounts of related data. The implementation has been rebased onto the latest master branch.

Implementation Summary

Core LINQ Changes Successfully Added:

  1. LargeDataOptimizedEntityFrameworkPersistenceProvider.cs - New optimized persistence provider that uses Entity Framework Core's AsSplitQuery() method to prevent Cartesian explosion in complex queries
  2. Entity Framework Version Upgrade - Updated netstandard2.1 target from EF Core 3.1.2 to 5.0.1 to support the AsSplitQuery() feature
  3. Enhanced PostgreSQL Extensions - Added largeDataOptimized boolean parameter to UsePostgreSQL() method
  4. Enhanced SQL Server Extensions - Added largeDataOptimized boolean parameter to UseSqlServer() method

Excluded Changes (As Requested):

  • All integration test generic parameter changes (<TSelf> additions)
  • Formatting/whitespace changes
  • Test collection name constant changes
  • Duplicated test scenarios for optimization testing
  • Any other non-LINQ related modifications from the original PR

Latest Updates:

  • Successfully rebased onto latest master branch (commit 76c011c)
  • Full solution builds successfully after rebase
  • All LINQ optimization changes preserved

Technical Details

Database Query Optimization: The new LargeDataOptimizedEntityFrameworkPersistenceProvider addresses the non-optimal database query issue (issue #1169) by using AsSplitQuery() in Entity Framework queries. This prevents the Cartesian product problem when loading workflows with many execution pointers and their related data.

Usage:

  • PostgreSQL: options.UsePostgreSQL(connectionString, canCreateDB, canMigrateDB, largeDataOptimized: true)
  • SQL Server: options.UseSqlServer(connectionString, canCreateDB, canMigrateDB, largeDataOptimized: true)

Files Changed:

  • LargeDataOptimizedEntityFrameworkPersistenceProvider.cs (new file, 135 lines)
  • WorkflowCore.Persistence.EntityFramework.csproj (EF version update)
  • PostgreSQL ServiceCollectionExtensions.cs (added optimization option)
  • SQL Server ServiceCollectionExtensions.cs (added optimization option)

This implementation provides the exact LINQ optimization benefits from PR #1302 while maintaining a minimal changeset focused purely on the database query performance improvements.

Original prompt

please recreate PR #1302 as a new PR, but without all the unnecessary changes. Only include the Linq changes, please exclude the changes to the integrations tests where generic parameters are being added. Also please leave out the formatting changes.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

…rovider with AsSplitQuery

Co-authored-by: danielgerlag <2357007+danielgerlag@users.noreply.github.com>
Copilot AI changed the title please recreate PR #1302 as a new PR, but without all the unnecessary changes. Only include the Linq changes, please exclude the changes to the integrations tests where generic parameters are being added. Also please leave out the formatting changes. Add LINQ database query optimizations for large data scenarios Sep 29, 2025
Copilot AI requested a review from danielgerlag September 29, 2025 02:13
Copilot finished work on behalf of danielgerlag September 29, 2025 02:13
@danielgerlag
Copy link
Owner

@copilot please rebase this PR from master

Copilot AI and others added 7 commits October 1, 2025 20:37
…thod

Co-authored-by: danielgerlag <2357007+danielgerlag@users.noreply.github.com>
Co-authored-by: danielgerlag <2357007+danielgerlag@users.noreply.github.com>
Co-authored-by: danielgerlag <2357007+danielgerlag@users.noreply.github.com>
Co-authored-by: danielgerlag <2357007+danielgerlag@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 1, 2025

@copilot please rebase this PR from master

Done! The PR has been successfully rebased onto the latest master branch (commit 76c011c). All LINQ optimization changes are preserved and the solution builds successfully. The rebased changes are now in commit b31d6b5.

Copilot finished work on behalf of danielgerlag October 1, 2025 20:38
@danielgerlag danielgerlag marked this pull request as ready for review October 3, 2025 23:31
@danielgerlag danielgerlag requested a review from glucaci as a code owner October 3, 2025 23:31
@danielgerlag danielgerlag merged commit 32c4c37 into master Oct 5, 2025
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants