Feature request: Migrations

[filidorwiese]

Hi! Materialize looks awesome! We're considering moving away from KSQL to Materialize for features like self-joins and Postgres driver compatibility. However, the one thing that is preventing us from adopting Materialize is some form of migrations tooling.

I probably won't have to explain to database developers the merits of migrations to handle schema updates on production environments. Could you shed some light on how people are currently using Materialize without it? Are there some plans to create a migrations tool in the near future? Or do people roll their own?

For sake of completeness, this is a link to the KSQL migration tool documentation, which in our experience does the job adequately.

[bobbyiliev]

Hi there @filidorwiese 👋

One option to do this with Materialize is with dbt. Here is link to the official documentation on how to use dbt to manage Materialize:

• How to use dbt to manage Materialize

Also, here is another helpful link for a quick introduction:

• Introducing: dbt + Materialize

I also have this demo that shows this in action together with Redpanda:

• Demo: How to use dbt with Materialize

Depending on your usecase, another option might be to use a side container that you could use to execute a startup SQL script to create your schema, here is a quick Docker-compose example with this in action:

• Materialize bootstrap SQL script example

Another option that I've tried in the past is to use a migration tool like the following, though the level of support for such migration tools might be limited at the moment but will improve as we extend the coverage of pg_catalog in Materialize (#2157):

• Laravel database migrations with Materialize

Hope that this helps!

Best,

Bobby

[filidorwiese]

Thank you, I'll look into those options