Skip to content

refactor(experimentation): extract a SQL dialect from the results queries - #8690

Draft
Zaimwa9 wants to merge 5 commits into
mainfrom
refactor/warehouse-sql-dialect
Draft

Zaimwa9 wants to merge 5 commits into
mainfrom
refactor/warehouse-sql-dialect

Conversation

@Zaimwa9

@Zaimwa9 Zaimwa9 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor
  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Fourth of the BYO warehouse series, following #8683. Prepares the results SQL for Databricks; no behaviour change.

  • warehouses/dialect.py: a minimal Dialect with one ClickHouseDialect. Function-level hooks only (param markers, list membership, counts, time buckets, conditional aggregates, bool to number) plus one clause hook for the conversions unnest.
  • ResultsQueryBuilder takes a dialect. The shared readers and QueryRunner move from clickhouse.py to warehouses/queries.py; clickhouse.py keeps only ClickHouse plumbing.
  • Managed Flagsmith and BYOW ClickHouse both pass ClickHouseDialect.

How did you test this code?

The first commit adds test_queries.py, pinning every statement both ClickHouse providers send, with the exact params and driver kwargs, captured from the code before the refactor. The refactor commits leave those expectations unchanged; the last commit drops the older fragment assertions in test_services.py they supersede.

Every statement both providers send was also captured with mocked drivers on main and this branch, and compared byte for byte (SQL, params, driver arguments, client settings): 38/38 calls identical, 77 statements, full capture abae9afbf8f1 on both.

Per-call comparison

Call Statements main pr4 Equal
BYOW: get_event_names 1 e5ef968bcce8 e5ef968bcce8 ✅
BYOW: get_event_stats 1 291482f32dd8 291482f32dd8 ✅
BYOW: get_exposure_buckets [day] 1 4fa2bfbe1625 4fa2bfbe1625 ✅
BYOW: get_exposure_buckets [hour] 1 ec4bcc987db5 ec4bcc987db5 ✅
BYOW: get_results_aggregates [day, count] 2 71a98687c89e 71a98687c89e ✅
BYOW: get_results_aggregates [day, every aggregation] 3 914c628ca308 914c628ca308 ✅
BYOW: get_results_aggregates [day, mean] 2 c42bada7e00a c42bada7e00a ✅
BYOW: get_results_aggregates [day, no metrics] 2 522e62c888d1 522e62c888d1 ✅
BYOW: get_results_aggregates [day, occurrence] 3 f2345e8222f3 f2345e8222f3 ✅
BYOW: get_results_aggregates [day, same event twice] 3 608b1d2ceba9 608b1d2ceba9 ✅
BYOW: get_results_aggregates [day, sum] 2 01f6ed39b3e3 01f6ed39b3e3 ✅
BYOW: get_results_aggregates [hour, count] 2 2dcd1adcaa6f 2dcd1adcaa6f ✅
BYOW: get_results_aggregates [hour, every aggregation] 3 58e36c180d66 58e36c180d66 ✅
BYOW: get_results_aggregates [hour, mean] 2 5328ecf33a05 5328ecf33a05 ✅
BYOW: get_results_aggregates [hour, no metrics] 2 0159a686cb73 0159a686cb73 ✅
BYOW: get_results_aggregates [hour, occurrence] 3 c851c214666b c851c214666b ✅
BYOW: get_results_aggregates [hour, same event twice] 3 f955ae0341ae f955ae0341ae ✅
BYOW: get_results_aggregates [hour, sum] 2 194864a1c277 194864a1c277 ✅
BYOW: verify 1 5aef87bd4dee 5aef87bd4dee ✅
managed: get_event_names 1 90d16c0474c8 90d16c0474c8 ✅
managed: get_event_stats 1 6100ba24e594 6100ba24e594 ✅
managed: get_exposure_buckets [day] 1 87ee052a01a7 87ee052a01a7 ✅
managed: get_exposure_buckets [hour] 1 c836415cee3b c836415cee3b ✅
managed: get_results_aggregates [day, count] 2 9748bf455c4c 9748bf455c4c ✅
managed: get_results_aggregates [day, every aggregation] 3 5691d95e679a 5691d95e679a ✅
managed: get_results_aggregates [day, mean] 2 e10c63e789bb e10c63e789bb ✅
managed: get_results_aggregates [day, no metrics] 2 4dc6062ac6a9 4dc6062ac6a9 ✅
managed: get_results_aggregates [day, occurrence] 3 74f522eb50e5 74f522eb50e5 ✅
managed: get_results_aggregates [day, same event twice] 3 18c3736f116f 18c3736f116f ✅
managed: get_results_aggregates [day, sum] 2 4d5a07e6841a 4d5a07e6841a ✅
managed: get_results_aggregates [hour, count] 2 feec10be563e feec10be563e ✅
managed: get_results_aggregates [hour, every aggregation] 3 0abc930ccac2 0abc930ccac2 ✅
managed: get_results_aggregates [hour, mean] 2 ffcd4530a9e6 ffcd4530a9e6 ✅
managed: get_results_aggregates [hour, no metrics] 2 c0a8bbd6b26f c0a8bbd6b26f ✅
managed: get_results_aggregates [hour, occurrence] 3 1c334f61553b 1c334f61553b ✅
managed: get_results_aggregates [hour, same event twice] 3 0d507fbf9565 0d507fbf9565 ✅
managed: get_results_aggregates [hour, sum] 2 bff1c7ffbdb0 bff1c7ffbdb0 ✅
managed: verify 0 b3150660859e b3150660859e ✅

@vercel

vercel Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
docs Ignored Ignored Preview Oct 7, 2026 12:27pm UTC
flagsmith-frontend-preview Ignored Ignored Preview Oct 7, 2026 12:27pm UTC
flagsmith-frontend-staging Ignored Ignored Preview Oct 7, 2026 12:27pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 4fca11a6-c97b-45a4-b779-41b6d16602c4
📥 Commits

Reviewing files that changed from the base of the PR and between fa52035 and cde1402.

📒 Files selected for processing (8)
  • api/experimentation/results_query.py
  • api/experimentation/warehouses/clickhouse.py
  • api/experimentation/warehouses/dialect.py
  • api/experimentation/warehouses/flagsmith.py
  • api/experimentation/warehouses/queries.py
  • api/tests/unit/experimentation/test_services.py
  • api/tests/unit/experimentation/warehouses/test_golden_sql.py
  • docs/docs/deployment-self-hosting/observability/_events-catalogue.md

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.


📝 Walkthrough

Walkthrough

The change adds a Dialect interface and a ClickHouse implementation for SQL expressions used by experimentation result queries. It moves warehouse query builders and result readers into a shared module, then updates both warehouse adapters to use those helpers with CLICKHOUSE_DIALECT. Tests now check the generated SQL and parameters for both providers. The event catalogue source locations are also updated.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to cde14

The warehouse-query refactor has no identified merge-blocking issue; it is ready for normal checks.

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added api Issue related to the REST API refactor labels Oct 7, 2026
@codecov

codecov Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.85%. Comparing base (fa52035) to head (8ab2a82).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8690   +/-   ##
=======================================
  Coverage   98.84%   98.85%           
=======================================
  Files        1663     1666    +3     
  Lines       68625    68696   +71     
=======================================
+ Hits        67835    67906   +71     
  Misses        790      790           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Zaimwa9

Zaimwa9 commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

@themis-blindfold review

@github-actions github-actions Bot added refactor and removed docs Documentation updates refactor labels Oct 7, 2026
@themis-blindfold

Copy link
Copy Markdown
Contributor

⚖️ Themis review: ✅ Ship it

TL;DR: No correctness issues found. The refactor preserves the emitted ClickHouse SQL and parameters across both managed and customer-backed warehouse paths, with golden coverage for each statement. Completed CI checks passed; a focused local rerun could not start because this sandbox blocks dependency downloads.

Area Score
🎯 Correctness 5/5
🧪 Test coverage 5/5
📐 Code quality 5/5
🚀 Product impact 3/5
📝 Walkthrough
  • SQL dialect boundary - moves warehouse-specific SQL fragments behind the new dialect while retaining the current ClickHouse implementation.
  • Query readers - centralises shared event, exposure, and experiment-result queries for both warehouse providers.
  • Regression coverage - pins every emitted ClickHouse statement, parameters, and driver options for both provider paths.
🧪 How to verify
  1. Run uv run pytest tests/unit/experimentation/warehouses/test_golden_sql.py -q from api/ to compare every warehouse statement and parameter set.
  2. Run uv run pytest tests/unit/experimentation/test_services.py -q from api/ to cover metric aggregation and conversion decoding.
  3. Exercise experiment results with no metrics, count-only metrics, and occurrence metrics against both the managed and a customer ClickHouse warehouse.
    Automate: Keep the golden SQL tests mandatory whenever a dialect or shared query changes.

Product take: No material product impact. This is useful enabling work for additional warehouse support while keeping current experiment reporting behaviour stable.

🧭 Assumptions & unverified claims

No unverified assumptions or claims.

The SQL moved house without changing its accent · reviewed at cde1402

@themis-blindfold

Copy link
Copy Markdown
Contributor

⚖️ Themis review: ✅ Ship it

TL;DR: This preserves the emitted ClickHouse SQL and driver parameters while moving the shared query construction behind a small dialect boundary. Completed CodeQL, Python analysis, pre-commit, and documentation-artifact checks passed; API unit and documentation link checks were still running.

Area Score
🎯 Correctness 5/5
🧪 Test coverage 4/5
📐 Code quality 5/5
🚀 Product impact 2/5
📝 Walkthrough
  • Dialect boundary - parameter, aggregate, bucketing, and unnest fragments now come from the ClickHouse dialect without changing the generated statements.
  • Shared warehouse reads - both managed and customer ClickHouse paths use the extracted query readers while retaining their existing client-specific execution settings.
  • Regression coverage - golden tests pin SQL, parameters, and driver kwargs for lookup, exposure, results, and conversion reads across both providers.
🧪 How to verify
  1. Run cd api && uv run pytest -q tests/unit/experimentation/warehouses/test_queries.py.
  2. Run cd api && uv run pytest -q tests/unit/experimentation/warehouses/test_clickhouse.py.
  3. Run cd api && uv run pytest -q tests/unit/experimentation/test_services.py.
  4. Exercise an experiment results refresh against both the managed event store and a customer ClickHouse connection, including occurrence and value metrics.
    Automate: keep the golden SQL tests as the regression gate whenever a new warehouse dialect is added.

Product take: Minor infrastructure work with no intended user-visible change; it makes the upcoming warehouse support safer to extend.

🧭 Assumptions & unverified claims
  • Focused tests were not run locally because the required dependencies were unavailable and could not be fetched in this environment.

The SQL has changed homes, not its habits · reviewed at 8ab2a82

This branch was successfully deployed

1 active (outdated) deployment
Preview – docs — cde1402e Deployed Oct 7, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant