Skip to content

Avoid calling getPivotValues() to improve PIVOT query performance #3937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Dec 28, 2022

Conversation

labkey-adam
Copy link
Contributor

@labkey-adam labkey-adam commented Dec 17, 2022

Rationale

In the absence of an explicit PIVOT ... IN clause, LabKey must execute portions of PIVOT queries to determine the resulting column list. Depending on the query, this can be very expensive; see https://www.labkey.org/OHSU/Support%20Tickets/issues-details.view?issueId=46881. This PR attempts to avoid calling getPivotValues() (the column-list query) unless it's absolutely required. This can improve performance significantly for cases that don't need the query results, e.g., rendering the schema browser tree and populating the linked schema table drop-down list. Approach:

  • AbstractTableInfo: introduce initializeColumns(), called immediately before columns are accessed for the first time. Adding columns in this method (instead of the constructor) avoids the potentially expensive operation in cases where the columns aren't used. Call ensureInitialColumnsAreAdded() from all methods that interact directly with _columnMap.
  • PivotTableInfo: populate the initial columns in initializeColumns() and defer validation (missing parameter values, etc.) to getAllColumns(). Rework the exception handling in getPivotValues() and its callers to ensure reasonably friendly error messages.

Schema browser tree now avoids executing PIVOT queries. GetQueriesAction (used in linked schema admin page) still indirectly invokes getPivotValues() (e.g., for URL handling). We should either parameterize GetQueriesAction further (includeUrls parameter, maybe false by default?) or switch linked schema admin page to call getSchemaTree.api instead of getQueries.api, to avoid the expensive query. Will address this in a follow-on PR.

@labkey-adam labkey-adam changed the title Delay calling getPivotValues() to improve performance Avoid calling getPivotValues() to improve performance Dec 17, 2022
@labkey-adam labkey-adam marked this pull request as ready for review December 21, 2022 18:34
@labkey-adam labkey-adam changed the title Avoid calling getPivotValues() to improve performance Avoid calling getPivotValues() to improve PIVOT query performance Dec 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants