Restructure list annotations query to guide Postgres to a more efficient query plan #8498
Conversation
📝 WalkthroughWalkthroughThis pull request documents a bug fix in the changelog for an annotation list loading issue and refactors the SQL query in the AnnotationDAO. The SQL query in the Changes
Possibly related PRs
Suggested labels
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (1)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
app/models/annotation/Annotation.scala (1)
407-443: Validate the query filtering and CTE usage.The new CTE (
an) cleanly gathers essential annotation properties and filters by state, access, user, and type. Ensuring indexes exist on_id,visibility,state,_user, and_datasetcolumns will help maintain good performance, especially on large tables.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
CHANGELOG.unreleased.md(1 hunks)app/models/annotation/Annotation.scala(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: circleci_build
🔇 Additional comments (3)
app/models/annotation/Annotation.scala (2)
402-406: Good approach to split the query.Separating the core “annotation” selection from the eventual team aggregation helps avoid fanout and allows applying the
LIMITearlier, which should yield a more efficient query plan in Postgres.
444-494:❓ Verification inconclusive
Consider verifying query performance with EXPLAIN and broad data.
While the refactoring is beneficial, we recommend verifying the execution plan and comparing timings in your typical staging environment. You can run an EXPLAIN (ANALYZE, BUFFERS) to confirm whether Postgres is leveraging the new CTE approach efficiently.
🏁 Script executed:
#!/bin/bash # Description: Run an EXPLAIN ANALYZE on the new query from lines 444-494. cat <<'EOF' > /tmp/annotation_query.sql EXPLAIN (ANALYZE, BUFFERS) WITH an AS ( ... -- same subquery from lines 407-443 ) SELECT ... -- same final SELECT from lines 444-494 EOF psql -h <YOUR_DB_HOST> -U <YOUR_DB_USER> -d <YOUR_DB_NAME> -f /tmp/annotation_query.sql rm /tmp/annotation_query.sqlLength of output: 184
Verify Query Performance Using Correct DB Connection Details
The refactoring looks promising; however, the EXPLAIN script ran with placeholder values (e.g., YOUR_DB_HOST, YOUR_DB_USER) and did not execute as intended. Please re-run the performance verification in your staging environment with the proper database connection parameters. This manual check is essential to ensure that Postgres leverages the new CTE approach efficiently.
- Replace placeholders with your actual DB host, user, and database name.
- Execute the script with correct parameters to get the EXPLAIN (ANALYZE, BUFFERS) output.
- Confirm that the query execution plan and timings meet your performance criteria before merging.
CHANGELOG.unreleased.md (1)
28-28: Thanks for documenting the fix.Adding this note about the annotated list loading fix is crucial for users tracking performance improvements and release notes.
…ent query plan (#8498) * Restructure list annotations query to guide Postgres to a more efficient query plan * changelog --------- Co-authored-by: MichaelBuessemeyer <39529669+MichaelBuessemeyer@users.noreply.github.com>
* Fix undo in merger mode (#8463) * rename some variables * more renaming * more renaming * fix undo in merger mode * format * change any to number --------- Co-authored-by: Michael Büßemeyer <frameworklinux+MichaelBuessemeyer@users.noreply.github.com> Co-authored-by: MichaelBuessemeyer <39529669+MichaelBuessemeyer@users.noreply.github.com> * Fix navbar menu highlighting (#8473) * fix highlighting the correct navbar item * update changelog * Release 25.03.0 * add highlights * fix changelog and migration guide for 25.03.0 * Restructure list annotations query to guide Postgres to a more efficient query plan (#8498) * Restructure list annotations query to guide Postgres to a more efficient query plan * changelog --------- Co-authored-by: MichaelBuessemeyer <39529669+MichaelBuessemeyer@users.noreply.github.com> * Fix concurrency bug in skeleton saving (#8513) * Fix concurrency bug in skeleton saving * changelog * update changelog and migration guide for release 25.03.1 --------- Co-authored-by: Michael Büßemeyer <frameworklinux+MichaelBuessemeyer@users.noreply.github.com> Co-authored-by: MichaelBuessemeyer <39529669+MichaelBuessemeyer@users.noreply.github.com> Co-authored-by: Tom Herold <tom.herold@scalableminds.com> Co-authored-by: Florian M <fm3@users.noreply.github.com>
2 participants
It appears the new postgres version doesn’t find an efficient query plan anymore for our old WITH query. This PR restructures the query to get the LIMIT in early.
Steps to test:
Issues: