chore(deps): Migrate from crate[sqlalchemy] to sqlalchemy-cratedb

[amotl]

SUMMARY

• The CrateDB SQLAlchemy dialect needs more love, so it was separated from the DBAPI HTTP driver.
• This patch concludes the update to the new designated CrateDB SQLAlchemy dialect package, sqlalchemy-cratedb, which has been validated on a few other downstream packages already.
• The patch also updates the documentation section about configuring the CrateDB connector driver (preview).

Personal Message

Dear Superset maintainers, first things first: Thanks a stack for conceiving and maintaining Apache Superset, you know who you are. Hereby, we are submitting a little update about the database adapter of CrateDB. With kind regards, Andreas.

May we ask you to approve running the software tests on this draft PR, so we can get an impression if anything breaks?

/cc @hammerhead, @hlcianfagna, @surister

[github-actions]

Congrats on making your first PR and thank you for contributing to Superset! 🎉 ❤️

We hope to see you in our Slack community too! Not signed up? Use our Slack App to self-register.

[rusackas]

Thanks for the PR! We very much appreciate it :) Running CI. 🤞

[john-bodley]

@amotl it seems likely you've contributed to both the crate SQLAlchemy dialect and sqlalchemy-cratedb—serving as the main author for the later. What was the main reason for splitting this work out, i.e., why isn't it viable to simply update the SQLAlchemy dialect for the former?

[amotl]

Hi @john-bodley. Thanks for asking.

What was the main reason for splitting this work out, i.e., why isn't it viable to simply update the SQLAlchemy dialect for the former?

The Python DB API HTTP driver is stable, and does not receive many changes any longer, actually mostly none. This is by far not the case with the SQLAlchemy dialect. To get an idea about the relevant traffic, let us enumerate the whole stack of requests and improvements already queuing up.

• https://github.com/crate/sqlalchemy-cratedb/issues
• https://github.com/crate/sqlalchemy-cratedb/pulls

In particular, that patch demonstrates that the HTTP driver is not necessarily tied to the SQLAlchemy dialect, and vice versa.

• Dialect: Add support for asyncpg and psycopg3 drivers crate/sqlalchemy-cratedb#11

In this spirit, we decided to split concerns between those two artefacts, by separating them into two different repositories.
Currently, the SQLAlchemy dialect still exclusively uses the HTTP driver, but it will give users a choice after the aforementioned patch has been integrated.

Let us know about any further questions you may have on this topic.

[amotl]

Hi. We just refreshed this PR, bumping the dependency spec to use the most recent sqlalchemy-cratedb package, and updating the corresponding documentation section about CrateDB.

[amotl]

Hi. While being at it, refreshing the canonical dependencies for the CrateDB connector, we took the chance to also update the relevant documentation section. We hope you agree with those minor updates being slapped into the same PR per aef7f2d.

@hlcianfagna, @proddata, @simonprickett, @kneth: Please also have a look at those updates, and suggest any improvements that come to mind. See GitHub-rendered preview.

[villebro]

LGTM, thanks for the detailed context on the evolution of the driver/dialect.

[amotl]

Hi. Thanks for merging. We also just updated the canonical migration page on our documentation. This page educates about the package switchover, not specifically related to Apache Superset.

• https://cratedb.com/docs/sqlalchemy-cratedb/migrate-from-crate-client.html

[villebro]

Thanks for all the work @amotl , this is definitely open source at it's finest ❤️

[amotl]

Thanks for all your excellent work on Superset! 💯