-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: Speed up create_hogql_database #27451
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense, though seems to have broken some tests. I assume the issues is that some tests flipping between PoE modes and other modifiers didn't discard self.database
somewhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
General principle sounds good!
To "yes and" this - I'd guess that most of the 0.3 seconds in create_initial_channel_type comes from parsing HogQL, if we had a good pattern for caching the results of parsing HogQL in general (before team-specific context or placeholder values were applied) we could probably find similar speedups in a bunch of trends and web analytics queries
Suspect IssuesThis pull request was deployed and Sentry observed the following issues:
Did you find this useful? React with a 👍 or 👎 |
Problem
create_hogql_database is slow (somewhere between 1-1.5 seconds per run for team 2), and we run it multiple times for a single query.
This should make every query we run 1-2 seconds faster.
Changes
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
Does this work well for both Cloud and self-hosted?
How did you test this code?