New Community Provider: ClickHouse #67151
Unanswered
BentsiLeviav
asked this question in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi Airflow community,
I'd like to propose adding a new community provider for ClickHouse.
Integration overview
ClickHouse is an open-source column-oriented database management system designed for online analytical processing (OLAP). It is widely used for real-time analytics, AI workloads, o11y and is particularly popular in data engineering pipelines where large volumes of data need to be queried fast. Many Airflow users already use ClickHouse as a destination or source in their data pipelines, and a first-class provider would make that integration simpler, more consistent, and easier to maintain.
The provider exposes a
ClickHouseHookthat extendsDbApiHookvia theclickhouse-connectHTTP client library, enabling full compatibility withSQLExecuteQueryOperatorand the rest of the Airflow SQL ecosystem out of the box.System tests
The provider integrates with a live ClickHouse service. System tests are included in the PR (
tests/system/clickhouse/example_clickhouse.py) and cover the full lifecycle: create table, insert, read rows, and drop table.Proposed stewards
Sponsoring committer(s)
@eladkal
Working implementation
#67080
Incubation commitment
We commit to:
Best regards,
Bentsi Leviav
Beta Was this translation helpful? Give feedback.
All reactions