Slow seeding when having a large seed file and an anonymous block. #6686

Activity

1. Rinze-Smits

   added

   🐛 Bug

   on Sep 21, 2026
2. linear-code

   added

   supabase/cli

   on Sep 21, 2026
3. iamsharduld commented on Sep 21, 2026

   

   iamsharduld

   on Sep 21, 2026

   More actions

   Reproduced on 2.117.0 and current develop. The statement splitter is quadratic in statement length. Root cause and a small fix below.

   Cause: splitRaw (apps/cli/src/command-internal/sql-split.ts) builds the current statement with acc += rune, which Bun stores as a rope. The states then slice two or more characters from near the end of acc. Each such slice copies the whole rope, so it costs O(statement length). That happens:

   • on every rune inside a $$/$tag$ body (DollarState) or a /* */ comment (BlockState)
   • on every c/C (isBeginAtomic) and $ (hasIdentifierRuneBefore) outside quotes and comments

   -- comments are unaffected because their delimiter is a single \n, and a one-character slice doesn't copy the rope. Migrations use the same splitter (parseMigrationContent in migration-file.ts), so a large DO $$ migration is just as slow.

   Fix: track offsets and slice the flat input instead of concatenating. Only splitRaw changes:

   let start = 0;
   let end = 0;
   for (const rune of sql) {
     end += rune.length;
     const acc = sql.slice(start, end);
     const next = state.next(rune, acc);
     if (next === null) {
       tokens.push({ text: acc, terminated: true });
       start = end;
       state = new ReadyState();
     } else {
       state = next;
     }
   }
   if (end > start) tokens.push({ text: sql.slice(start), terminated: false });

   Results (Bun 1.4.1, Apple M1 Pro):

   splitAndTrim, one statement	256 KB	512 KB	1 MB
   DO $$ … $$, before → after	1,886 → 10 ms	6,177 → 20 ms	21,897 → 27 ms
   /* … */, before → after	2,022 → 10 ms	6,299 → 18 ms	22,098 → 32 ms

   End to end, supabase db reset with a 0.95 MiB DO $$ seed takes about 22 s on both 2.117.0 and develop, and 0.67 s with the fix. The same block as a migration takes 0.69 s with the fix. Database state and CLI output are identical.

   sql-split.unit.test.ts passes unchanged. splitSqlTokens output also matches the current implementation on edge cases (nested block comments, tagged dollar quotes, BEGIN ATOMIC, doubled quotes, non-BMP characters, unterminated trailing statements) and on all the slow inputs.

   Happy to open a PR with this and a regression test. Could this get the open-for-contribution label?
4. 7ttp

   self-assigned this

   on Sep 21, 2026
5. 7ttp

   closed this as completedin #6697on Sep 23, 2026
6. pull

   added a commit that references this issue on Sep 23, 2026

   fix(cli): stop large sql blocks stalling `db reset` (CLI-2469) (supab…

   ...

   fd662d1