Add new operator for ClickHouse

[bharatnc]

Description

Add a new operator for ClickHouse.

Use case / motivation

Use case similar to any other operators like Postgres, MySQL etc for working with ClickHouse.

Related Issues

As far as I have looked, there is already a plugin that can be installed.

But would be nice to have this part of airflow alongside other available operators.

If this feature sounds acceptable, then I can work on adding this to airflow (please feel free to assign this issue to me). However, I'm not sure if a similar feature has already been requested before (as far as I have searched through the issues, I didn't find one). Thank you!

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[potiuk]

Sure thing ! Go ahead @bharatnc

[mik-laj]

Should we use DB API? Thanks to this, we won't have to implement operators for this software, but we will be able to use generic operators for SQL.

[potiuk]

I think there is a value in using specific operators for each DB. There is not a lot of code and DBAPI is designed rather as a very low-level API that usually is implemented by driver specific for the Database. This is how most of the DB operators work and allow to implement specific features of each operator with the API's that are much easier to use.

[bharatnc]

Thank you! I haven't taken a look at DB generic API but sounds like what @potiuk describes is true and applicable here. The generic DB API will work for basic queries but a lot of operations and other queries used in Clickhouse have rather specific syntax / functionality unique to that database. The overlap of functionality b/w generic DB API operations and Clickhouse operations are rather not very extensive. So I think it would be better to have a separate operator for this use case so that operations can be made more extensive in future. Do let me know if I am correct or if I've completely missed the intention here.

[potiuk]

Regardless if we would like to implement DBAPI also for ClickHouse it is not a good idea to do it now IMHO. If we decide to do that - we should do it as a separate PR and see how much of the commonalities we can extract at all. But I really do not think adding db api when adding new database (and much different than the regular one) is a good idea now. @mik-laj let us know if you think otherwise, but please go ahead @bharatnc without it.

[eladkal]

For anyone who wants to pick this task:
Python driver that can be used: https://github.com/mymarilyn/clickhouse-driver
Reference for PR that added another influxdb provider #17068