Skip to content

Commit b19656e

Browse files
committed
fix(service-analytics): the unknown dialect arm folds $icontains with a portable construct, not translate()
`normalizeSqlDialect` routes EVERYTHING it cannot name onto `unknown` — an unset hook, `'oracle'`, `'libsql'`, a `SqlDriver` given a class client. #15780 left that arm folding with `translate()` on the reading that it "was never broken", which held only for the dialects the arm was pictured as (mssql, oracle). SQLite reaches it through four embedder compositions, and SQLite has no `translate()`, so all three of this package's compilers emitted a statement the engine refuses — a documented operator answering 500 because one OPTIONAL field was left out. The arm now folds with one nested `REPLACE` per ASCII letter: the same chain the MySQL arm already used, minus its `CAST(… AS BINARY)`, so there is one builder and the two cannot fold different alphabets. It parses on every SQL dialect and is ASCII-only BY CONSTRUCTION, so it serves both families `unknown` conflates — PostgreSQL/Oracle-like keep `translate()`'s exact result set, SQLite-like get an answer at all. - `postgres` is untouched and byte-identical; so is the case-EXACT family on every arm (`fold` false makes the fold function the identity on both). - ⛔ Not `LOWER()`, which `driver-sql`'s own `unknown` arm uses: it follows the collation and would trade this parse failure for silently wrong rows on PostgreSQL, the Unicode fold #4706 Q1 = A rules out. - The pins that moved are re-aimed at the property, not regenerated, and the pre-fix bytes are kept as a control that the engine still refuses them. - `SqliteWasmDriver`'s `isSqlite` override — 0 direct test hits, the sole reason no in-repo SQLite driver lands on this arm — gets a direct pin, with a control showing the base class answers `unknown` for that very config. Refs #16028 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XpTx2tbq3pZRYAdoGt6E6Y
1 parent bdc0218 commit b19656e

9 files changed

Lines changed: 409 additions & 54 deletions

File tree

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#16028] The `isSqlite` override, pinned DIRECTLY — the one thing that makes
5+
* this transport answer `"sqlite"` when something outside the driver asks which
6+
* SQL it speaks.
7+
*
8+
* ## Why this file exists
9+
*
10+
* `SqlDriver.dialectName` is public for exactly one consumer:
11+
* `service-analytics` compiles its own statements (an analytics `where`, an
12+
* ADR-0021 D-C read scope, the `/analytics/sql` echo) and needs the same
13+
* per-dialect construct choices the driver makes. Answering `'sqlite'` is what
14+
* routes `$icontains` onto `lower(col) GLOB lower(?)`; answering `'unknown'`
15+
* routes it onto the residue arm instead.
16+
*
17+
* The base class derives that answer by STRING-MATCHING `config.client` against
18+
* {@link SqlDriver}'s emission sets — and this transport passes a knex Client
19+
* CLASS, which is no string at all. So the correct answer here is produced by
20+
* one three-line override and by nothing else.
21+
*
22+
* ⚠️ #16028 measured that override at **0 direct test hits**: the only cover was
23+
* an indirect row-set pin (#15684), which would keep passing if the override
24+
* moved, because the ROWS come out the same either way — the driver runs its own
25+
* SQL through its own SQLite. What changes silently is the answer handed to a
26+
* package that compiles SQL for a DIFFERENT engine. That is the gap this file
27+
* closes, and it is the reason the control below is not decoration: it shows the
28+
* base class answering `'unknown'` for this exact config, so the pin above is a
29+
* measurement of the override rather than of the class hierarchy.
30+
*/
31+
32+
import { describe, it, expect, afterEach } from 'vitest';
33+
import { mkdtempSync, rmSync } from 'node:fs';
34+
import { tmpdir } from 'node:os';
35+
import { join } from 'node:path';
36+
import { SqlDriver } from '@objectstack/driver-sql';
37+
38+
import { SqliteWasmDriver } from '../src/index.js';
39+
40+
/** Nothing here connects — but every knex instance built is still torn down. */
41+
const opened: Array<{ disconnect(): Promise<void> }> = [];
42+
const dirs: string[] = [];
43+
44+
afterEach(async () => {
45+
await Promise.all(opened.splice(0).map((d) => d.disconnect().catch(() => {})));
46+
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true });
47+
});
48+
49+
const track = <T extends { disconnect(): Promise<void> }>(d: T): T => {
50+
opened.push(d);
51+
return d;
52+
};
53+
54+
describe('[#16028] SqliteWasmDriver names its dialect', () => {
55+
it('answers "sqlite" — the answer service-analytics compiles against', () => {
56+
// Read WITHOUT connecting, deliberately: `service-analytics` asks this
57+
// while BUILDING a statement, so an answer that needed a live pool would
58+
// arrive after the SQL it decides.
59+
expect(track(new SqliteWasmDriver({ filename: ':memory:' })).dialectName).toBe('sqlite');
60+
});
61+
62+
it('…on a file-backed database too, and with persistence on', () => {
63+
const dir = mkdtempSync(join(tmpdir(), 'wasm-dialect-'));
64+
dirs.push(dir);
65+
const file = join(dir, 'test.db');
66+
expect(track(new SqliteWasmDriver({ filename: file })).dialectName).toBe('sqlite');
67+
expect(track(new SqliteWasmDriver({ filename: file, persist: 'on-write' })).dialectName).toBe('sqlite');
68+
});
69+
70+
it('the client is a CLASS, so no string table could have answered it', () => {
71+
// The override's premise, asserted rather than assumed: if this ever became
72+
// a string knex spelling, the base class would answer on its own and the
73+
// override would be dead code rather than the load-bearing line it is.
74+
const client = (SqliteWasmDriver.toKnexConfig({ filename: ':memory:' }) as { client: unknown }).client;
75+
expect(typeof client).toBe('function');
76+
expect(typeof client).not.toBe('string');
77+
});
78+
79+
it('CONTROL: the base class answers "unknown" for this very config', () => {
80+
// Delete `isSqlite` from the subclass and this is what `service-analytics`
81+
// would be told — #16028's residue arm, which for `$icontains` emitted a
82+
// statement SQLite cannot parse at all until that card. This is what makes
83+
// the pin above a measurement of the OVERRIDE rather than of the hierarchy.
84+
const base = track(new SqlDriver(SqliteWasmDriver.toKnexConfig({ filename: ':memory:', pool: { min: 0, max: 1 } })));
85+
expect(base.dialectName).toBe('unknown');
86+
});
87+
});

‎packages/services/service-analytics/src/__tests__/icontains-dialect-sql.test.ts‎

Lines changed: 181 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -30,14 +30,27 @@
3030
* - **postgres → byte-identical text.** `translate()` is correct there today,
3131
* so the correct diff is no diff: the emitted SQL and params are asserted
3232
* verbatim against the pre-#15780 bytes, for all three compilers.
33-
* - **`unknown` (no hook wired) → byte-identical text, ALSO `translate()`.**
34-
* The residue keeps the shape it has always had. It is not an endorsement:
35-
* it is what still runs on the dialects nothing here models (mssql, oracle,
36-
* which do have `translate()`), and falling back to `LOWER()` for it would
37-
* silently restore the Unicode fold #4706 Q1 = A took away. Note this
38-
* DIVERGES from `driver-sql`'s `unknown` arm, which folds with `LOWER()` —
39-
* each side keeps its own pre-existing residue, and neither claims the
40-
* other's.
33+
* - **[#16028] `unknown` (no hook wired) → the portable `REPLACE` chain,
34+
* EXECUTED.** #15780 left this arm on `translate()` and called it
35+
* unbroken, which held only for the dialects the arm was PICTURED as
36+
* (mssql, oracle, which have `translate()`). `unknown` is not a dialect:
37+
* `normalizeSqlDialect` routes EVERYTHING it cannot name here, SQLite
38+
* included, so the residue arm carried the whole defect forward for the
39+
* four embedder compositions #16028 lists — among them a public
40+
* `AnalyticsService` constructed with its OPTIONAL `sqlDialect` left out.
41+
* The arm now folds with one nested `REPLACE` per ASCII letter, which every
42+
* SQL dialect parses, and the rows it answers on this engine are asserted
43+
* below from the same shared table the `sqlite` arm answers.
44+
*
45+
* ⛔ NOT `LOWER()`, which is what `driver-sql`'s `unknown` arm folds with:
46+
* `LOWER()` follows the collation, so adopting it would trade this parse
47+
* failure for SILENTLY wrong rows on PostgreSQL — the Unicode fold #4706
48+
* Q1 = A rules out. ⚠️ And measuring `LOWER()` HERE proves nothing about
49+
* that, because SQLite's `lower()` is ASCII-only and answers the shared
50+
* table correctly: the trap is a green SQLite reading standing in for a
51+
* PostgreSQL one, so the `REPLACE` chain is pinned as ASCII-only BY
52+
* CONSTRUCTION (over every ASCII code point, below) rather than by a fold
53+
* that happens to behave on the one engine in this container.
4154
* - **mysql → NOT MEASURED.** The nested-`REPLACE`-over-`CAST(… AS BINARY)`
4255
* arm is asserted as TEXT only. No MySQL server is provisionable in this
4356
* container — the same declared skip `driver-sql`'s #6518 suite and this
@@ -119,40 +132,118 @@ const ctxFor = (dialect?: string): DatasetScopedStrategyContext =>
119132
const TRANSLATE_FOLD =
120133
"translate(name, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz')";
121134

135+
/**
136+
* [#16028] The `unknown` arm's fold, rebuilt HERE from the ruled domain rather
137+
* than imported from the implementation — a second construction of the expected
138+
* text, so a change to the emitter's loop does not quietly re-bless itself.
139+
*
140+
* The endpoints are pinned verbatim beside every use (`REPLACE(name, 'A', 'a')`
141+
* innermost, `'Z', 'z')` outermost), which is also what discriminates this arm
142+
* from MySQL's: that one's innermost operand is `CAST(name AS BINARY)`.
143+
*/
144+
const ASCII_UP = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
145+
const ASCII_LO = 'abcdefghijklmnopqrstuvwxyz';
146+
const replaceFold = (expr: string): string => {
147+
let out = expr;
148+
for (let i = 0; i < ASCII_UP.length; i++) out = `REPLACE(${out}, '${ASCII_UP[i]}', '${ASCII_LO[i]}')`;
149+
return out;
150+
};
151+
122152
describe('[#15780] the compiled TEXT, per dialect — all three compilers', () => {
123153
const nativeSql = async (where: unknown, dialect?: string) =>
124154
new NativeSQLStrategy().generateSql(query(where), ctxFor(dialect));
125155
const echoSql = async (where: unknown, dialect?: string) =>
126156
new ObjectQLStrategy().generateSql(query(where), ctxFor(dialect));
127157

128-
it('postgres and a host that wired NO hook keep the pre-#15780 bytes exactly', async () => {
129-
// The non-regression half, for the two dialects that were already correct.
130-
// Asserted verbatim, not by shape: `translate()` is right on Postgres, and
131-
// the `unknown` residue must keep the only fold that still runs there.
132-
for (const dialect of [undefined, 'postgres'] as const) {
158+
it('postgres keeps the pre-#15780 bytes exactly — the arm that was always right', async () => {
159+
// The non-regression half. Asserted verbatim, not by shape: `translate()`
160+
// is correct on Postgres, so the correct diff there is no diff — through
161+
// #15780 and through #16028 alike.
162+
const out = await nativeSql({ name: { $icontains: 'acme' } }, 'postgres');
163+
expect(out.sql).toContain(
164+
`WHERE ${TRANSLATE_FOLD} LIKE translate($1, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') ESCAPE $2`,
165+
);
166+
expect(out.params).toEqual(['%acme%', '\\']);
167+
168+
const echo = await echoSql({ name: { $icontains: 'acme' } }, 'postgres');
169+
expect(echo.sql).toContain(
170+
`${TRANSLATE_FOLD} LIKE translate($1, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') ESCAPE $2`,
171+
);
172+
expect(echo.params).toEqual(['%acme%', '\\']);
173+
174+
expect(
175+
compileScopedFilterToSql({ name: { $icontains: 'acme' } } as FilterCondition, 't', { dialect: 'postgres' }),
176+
).toEqual({
177+
sql:
178+
`translate("t"."name", 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') LIKE ` +
179+
`translate(?, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') ESCAPE ?`,
180+
params: ['%acme%', '\\'],
181+
});
182+
});
183+
184+
it('[#16028] a host that wired NO hook folds with the PORTABLE chain, on all three compilers', async () => {
185+
// The moved cells, stated as the pin rather than left to a regenerate: the
186+
// `unknown` arm's emitted text CHANGED, from `translate()` — which SQLite
187+
// and MySQL/MariaDB cannot parse — to a nested `REPLACE` per ASCII letter,
188+
// which every SQL dialect can. The rows this text answers are executed
189+
// further down; here it is the bytes and the bindings.
190+
for (const dialect of [undefined, 'oracle', 'libsql'] as const) {
191+
// All three normalize to `unknown`: an unset hook, and two spellings
192+
// `normalizeSqlDialect` does not model. One arm, reached three ways.
133193
const out = await nativeSql({ name: { $icontains: 'acme' } }, dialect);
134194
expect(out.sql, String(dialect)).toContain(
135-
`WHERE ${TRANSLATE_FOLD} LIKE translate($1, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') ESCAPE $2`,
195+
`WHERE ${replaceFold('name')} LIKE ${replaceFold('$1')} ESCAPE $2`,
136196
);
197+
expect(out.sql, String(dialect)).not.toMatch(/translate\(/);
198+
// The chain's endpoints, verbatim — and the innermost operand is the bare
199+
// column, which is what tells this arm apart from MySQL's `CAST(… AS
200+
// BINARY)` one.
201+
expect(out.sql, String(dialect)).toContain("REPLACE(name, 'A', 'a')");
202+
expect(out.sql, String(dialect)).toContain("'Z', 'z')");
203+
expect(out.sql, String(dialect)).not.toMatch(/CAST\(/);
137204
expect(out.params, String(dialect)).toEqual(['%acme%', '\\']);
138205

139206
const echo = await echoSql({ name: { $icontains: 'acme' } }, dialect);
140207
expect(echo.sql, `echo ${String(dialect)}`).toContain(
141-
`${TRANSLATE_FOLD} LIKE translate($1, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') ESCAPE $2`,
208+
`${replaceFold('name')} LIKE ${replaceFold('$1')} ESCAPE $2`,
142209
);
210+
expect(echo.sql, `echo ${String(dialect)}`).not.toMatch(/translate\(/);
143211
expect(echo.params, `echo ${String(dialect)}`).toEqual(['%acme%', '\\']);
144212
}
145213

214+
// The read scope — the compiler where an unevaluable statement is an
215+
// ADR-0021 policy that cannot be applied at all.
146216
const noHook = compileScopedFilterToSql({ name: { $icontains: 'acme' } } as FilterCondition, 't');
147217
expect(noHook).toEqual({
148-
sql:
149-
`translate("t"."name", 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') LIKE ` +
150-
`translate(?, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') ESCAPE ?`,
218+
sql: `${replaceFold('"t"."name"')} LIKE ${replaceFold('?')} ESCAPE ?`,
151219
params: ['%acme%', '\\'],
152220
});
221+
expect(noHook.sql).not.toMatch(/translate\(/);
153222
expect(
154-
compileScopedFilterToSql({ name: { $icontains: 'acme' } } as FilterCondition, 't', { dialect: 'postgres' }),
223+
compileScopedFilterToSql({ name: { $icontains: 'acme' } } as FilterCondition, 't', { dialect: 'nonesuch' }),
155224
).toEqual(noHook);
225+
226+
// ⛔ And the arm did NOT become Postgres's: the two are now different text,
227+
// which is the whole point of splitting them.
228+
const pg = await nativeSql({ name: { $icontains: 'acme' } }, 'postgres');
229+
const unknown = await nativeSql({ name: { $icontains: 'acme' } }, undefined);
230+
expect(unknown.sql).not.toBe(pg.sql);
231+
expect(unknown.params).toEqual(pg.params);
232+
});
233+
234+
it('[#16028] the case-EXACT four are byte-identical on `unknown` — only the FOLD moved', async () => {
235+
// The blast-radius pin. `fold` is the only thing #16028 changed on this
236+
// arm, so every case-exact operator must still emit the plain `LIKE` it
237+
// emitted before — no `REPLACE` anywhere near them.
238+
for (const op of ['$contains', '$notContains', '$startsWith', '$endsWith'] as const) {
239+
const out = await nativeSql({ name: { [op]: 'acme' } }, undefined);
240+
expect(out.sql, op).not.toMatch(/REPLACE\(|translate\(|lower\(/);
241+
expect(out.sql, op).toMatch(/name (NOT )?LIKE \$1 ESCAPE \$2/);
242+
}
243+
expect(compileScopedFilterToSql({ name: { $contains: 'acme' } } as FilterCondition, 't')).toEqual({
244+
sql: '"t"."name" LIKE ? ESCAPE ?',
245+
params: ['%acme%', '\\'],
246+
});
156247
});
157248

158249
it('sqlite compiles lower() over GLOB — one bound value, no ESCAPE clause', async () => {
@@ -274,16 +365,81 @@ describe('[#15780] the three compilers, EXECUTED on a real SQLite engine', () =>
274365
]);
275366
});
276367

277-
it('the defect, still reachable through a host that answers no dialect — so these pins discriminate', async () => {
278-
// Before-red, stated as the engine's own refusal rather than argued: the
279-
// `unknown` residue still emits `translate()`, and this engine cannot parse
280-
// it. This is the control that keeps every green assertion above honest.
368+
it('[#16028] the PRE-fix bytes are still refused by this engine — the control the greens rest on', () => {
369+
// Before-red, kept as a control rather than deleted with the defect. This
370+
// is the statement the `unknown` arm emitted until #16028, rebuilt here and
371+
// handed to the engine: it does not parse. Every green below is therefore a
372+
// measurement of the CHANGE, not of an engine that would have accepted
373+
// anything.
374+
const preFix =
375+
`SELECT "id" FROM "rows" WHERE ${TRANSLATE_FOLD} LIKE ` +
376+
`translate(?, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') ESCAPE ?`;
377+
expect(() => run(preFix, ['%acme%', '\\'])).toThrow(/no such function: translate/);
378+
});
379+
380+
it('[#16028] a host that answers NO dialect now compiles a statement this engine RUNS', async () => {
381+
// The card's headline, executed: with no `sqlDialect` hook wired — the
382+
// shape a directly-constructed public `AnalyticsService` has when its
383+
// OPTIONAL field is omitted — the compiled `where` used to be refused by
384+
// the engine. It now parses, and it answers the shared table's rows.
281385
const { sql, params } = await new NativeSQLStrategy().generateSql(
282386
query({ name: { $icontains: 'acme' } }),
283387
unawareCtx,
284388
);
285-
expect(sql).toContain('translate(');
286-
expect(() => run(sql, params)).toThrow(/no such function: translate/);
389+
expect(sql).not.toMatch(/translate\(/);
390+
expect(run(sql, params)).toEqual(['1', '2']);
391+
});
392+
393+
it('[#16028] the `unknown` arm answers the shared $icontains table — all three compilers', async () => {
394+
// The same rows the `sqlite` arm is required to answer, required of the
395+
// residue arm. Executed on sql.js 1.14.1, the engine `driver-sqlite-wasm`
396+
// runs — which is what an `unknown` dialect that is really SQLite IS.
397+
for (const c of NAME_ICONTAINS) {
398+
expect(await executedIds(c.filter, unawareCtx), c.name).toEqual([...c.expected]);
399+
400+
const scoped = {
401+
...unawareCtx,
402+
getReadScope: (object: string) => (object === 'rows' ? (c.filter as FilterCondition) : null),
403+
} as DatasetScopedStrategyContext;
404+
const viaScope = await new NativeSQLStrategy().generateSql(query(undefined), scoped);
405+
expect(viaScope.sql, `read scope: ${c.name}`).not.toMatch(/translate\(/);
406+
expect(run(viaScope.sql, viaScope.params), `read scope: ${c.name}`).toEqual([...c.expected]);
407+
408+
const echo = await new ObjectQLStrategy().generateSql(query(c.filter), unawareCtx);
409+
expect(echo.sql, `echo: ${c.name}`).not.toMatch(/translate\(/);
410+
expect(run(echo.sql, echo.params), `echo: ${c.name}`).toEqual([...c.expected]);
411+
}
412+
});
413+
414+
it('[#16028] the portable chain folds ASCII and NOTHING else — every code point, executed', () => {
415+
// The property the arm rests on, measured instead of argued, because the
416+
// one thing that must never happen here is a fold that reaches past `A`-`Z`
417+
// (#4706 Q1 = A). Two halves:
418+
//
419+
// 1. the chain equals the simultaneous `A`-`Z` map — i.e. applying the 26
420+
// REPLACEs IN SEQUENCE cannot cascade, because every step writes a
421+
// lower-case letter and every later step matches an upper-case one;
422+
// 2. it touches nothing else — no accented letter, and none of the LIKE
423+
// metacharacters the escaping depends on staying literal.
424+
const asciiLower = (v: string) =>
425+
v.replace(/[A-Z]/g, (ch) => ASCII_LO[ASCII_UP.indexOf(ch)]);
426+
const probes = [
427+
...Array.from({ length: 128 }, (_, i) => String.fromCharCode(i)).filter((ch) => ch !== '\0'),
428+
'CAFÉ', 'café', 'ÀÉÎÕÜ', 'ÄÖÜ', 'ǍǏǑ', 'ΑΒΓ', 'АБВ', 'İIı',
429+
'ACME Corp', '100% match', 'a_b', 'A\\B', 'ZzAa',
430+
];
431+
for (const probe of probes) {
432+
const literal = probe.replace(/'/g, "''");
433+
const got = db.exec(`SELECT ${replaceFold(`'${literal}'`)}`)[0].values[0][0];
434+
expect(got, JSON.stringify(probe)).toBe(asciiLower(probe));
435+
}
436+
// ⚠️ And the control that says why `LOWER()` may not be adopted here even
437+
// though it passes on THIS engine: SQLite's `lower()` is ASCII-only, so it
438+
// agrees with the chain on every probe above. It is PostgreSQL's, which is
439+
// collation-aware, that would fold `É` — and no PostgreSQL server is
440+
// provisionable in this container. So this cell is why the arm is chosen by
441+
// CONSTRUCTION and not by what happens to pass locally.
442+
expect(db.exec(`SELECT lower('CAFÉ')`)[0].values[0][0]).toBe('cafÉ');
287443
});
288444

289445
it('NativeSQLStrategy answers the shared table\'s $icontains rows', async () => {

0 commit comments

Comments
 (0)