v2.9.4 Breaking change question

[Kygri]

Hi,
I have a few questions regarding the breaking change in v2.9.4.
Background:
I use the chainlit-datalayer repository .
I noticed that this schema doesn't have a table named steps only Step.
In the Thread table, there is a steps column that references the Step table.
I assumed, that the column rows should be in the "Steps" table, so I added the modes column with type Json? to the "Step" table and created a migration.sql file.

Chainlit runs without an error, but I noticed, that the modes column is always null, even though i implemented and used the modes feature.

Questions:

1. Do I really need to have a Steps table? If not, should the command be as follows:
   ALTER TABLE "Step" ADD COLUMN IF NOT EXISTS "modes" JSONB;
2. Is it normal that modes column is not populated, even though I have created a model selection as in the PR #2734?

[dosubot]

Hi @Kygri! I'm Dosu and I’m helping the chainlit team.

You don't need a separate "Steps" table—the singular "Step" table in your schema is correct for migrations. The right migration command for your setup is:

ALTER TABLE "Step" ADD COLUMN IF NOT EXISTS "modes" JSONB;

If the "modes" column is always null, it's because the migration only adds the column; it doesn't populate it. The application logic must explicitly serialize and save the modes data. In the SQLAlchemy data layer, you need to update the code that inserts steps so it serializes the modes dictionary before saving. For example, in the create_step() function, add:

parameters["modes"] = json.dumps(step_dict.get("modes", {}))

This ensures the modes field is stored as JSON in the database. For reference, see the discussion and fix in issue #2750. After making this change, verify that your modes selections are being persisted as expected.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[dominpm]

I think I'll have the same issue, currently on v2.9.0 and planning to update to latest, but I am not too sure about the breaking change in v2.9.4.

@Kygri were you able to fix it? And since you are also sing chainlit_datalayer and if you don't mind me asking, how do you do the migrations required? (Any other migrations not present in the official repo?)

Thanks!

[Kygri]

Hi @dominpm,
Here’s a overview of the migration path I followed when upgrading the Chainlit Datalayer. Please note: these are the steps that worked for me — your setup might differ, so always back up your database first or use a test environment to avoid data loss.
As the dosubot has already noted, the ‘modes’ column is not yet filled in. I am therefore unable to verify whether it is functioning correctly. I only tried it out on a test environment because I'm not familiar enough with it to be able to safely undo the changes if necessary.

Step 1: Add modes to Step table
In your schema.prisma file add the new modes field to the Step model. For example in line 97:
modes Json?

Step 2: Create the migration file

Note: your postgres database should be running

Export your DATABASE_URL. Example schema for linux: export DATABASE_URL="postgresql://$POSTGRES_USER:$POSTGRES_PASSWORD@0.0.0.0:$POSTGRES_PORT/$POSTGRES_DB"
Than run the migration in dev mode:

npx prisma@6.4.1 migrate dev

Your change should be registered and you will be prompted to name the migration. I called it: add modes to step.
Under migrations should appear a new folder.
The migration file in the folder should be similar to this:

-- AlterTable
ALTER TABLE "Step" ADD COLUMN     "modes" JSONB;

The column modes should now also be in your Step table in your database.

(optional) Step 3: Deploy
For Deploying the changes for example on prod, you need to deploy the changes

npx prisma@6.4.1 migrate deploy

[dominpm]

Thanks for the detailed explanation! Helps a lot.

I understand we will have to eventually migrate to the sqlalchemy data layer, as I have seen a pr being made for this.

Thanks!

[JamesLloydPartingtonSerendipityai]

I have a PR Chainlit/chainlit-datalayer#17 for them to add this change

[github-actions]

This issue is stale because it has been open for 14 days with no activity.