You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Pipeline is insert-only; all mutations move to raw admin SQL #158
Lock the ingest pipeline to inserts only. Every mutation other than INSERT — DELETE, UPDATE, TRUNCATE, DROP, ALTER, REPLACE, etc. — must be issued through POST /v1/query, which already gates on the admin/service role (or a policy role with RawSQL: true).
Why
Our policy engine authorizes mutations by evaluating row/column rules and JWT-claim check clauses against the payload of an operation (the columns being written). That model works cleanly for inserts; it does not work for predicate-driven mutations:
We have no way to prove a WHERE predicate is satisfiable only for rows the caller's role is allowed to touch — no row-level filter injection on DML.
Blast radius is unbounded — a single WHERE 1=1 (or a missing WHERE on DROP/TRUNCATE) wipes a table.
Cache-tag invalidation is straightforward for insert and for table-scoped mutations issued through raw SQL (we already extract mutation targets there); a partially-authorized predicate path would muddy that contract.
Rather than ship a partial enforcement story, we narrow the pipeline to the one shape policy can authorize today (insert) and route everything else through the existing admin-only raw SQL gate. The policy contract stays honest — "pipelined endpoints are policy-enforced; raw SQL is admin-only" — and the WHERE-authorization design is deferred without blocking alpha.
Scope — what to remove / change
internal/ingest/bento.go: drop the inline action: "delete" branch (DELETE FROM <table> WHERE id = ?) and its associated DLQ delete-envelope shape. Collapse the action check to "anything other than insert/empty → reject".
internal/ingest/bento_test.go: remove the delete-action coverage (drain, DLQ routing, unsafe-table, double-ack) — unreachable once the branch is gone.
Docs (docs/src/content/docs/architecture.md, api.md): drop the delete-envelope DLQ description and the Wave-DLQ-Type: delete-envelope header note. State explicitly that the ingest pipeline is insert-only and that all other mutations (DELETE/UPDATE/TRUNCATE/DROP/ALTER/REPLACE) require POST /v1/query under an admin-equivalent role.
/v1/query mutation-response shape: confirm DDL/DML statements that return no rows from ClickHouse marshal to HTTP 200 with [], not 500 (root cause of bug: TRUNCATE via /v1/query returns HTTP 500 #118). Likely needs the handler to route mutations through Exec rather than Query, or to tolerate a nil/empty ColumnTypes() result.
CHANGELOG: breaking-change entry — action: "delete" envelopes on ingest.<table> subjects are no longer honored; the ingest pipeline accepts inserts only. Use POST /v1/query with an admin role for any other mutation.
Confirm no other pipelined mutation path exists: /v1/tables/{table}/query AST (internal/query/ast.go), named pipes (internal/pipes), structured query handler. Today these look read-only; verify before closing.
Out of scope (deferred)
Re-introducing pipelined / structured mutations once the policy engine can authorize predicates. Track separately if/when scheduled.
Closes bug: TRUNCATE via /v1/query returns HTTP 500 #118 — TRUNCATE via /v1/query returning HTTP 500 is the same class of bug (a mutation statement through the raw-SQL handler not returning a clean success shape). Since /v1/query becomes the only sanctioned surface for non-insert mutations, fixing that response path is part of this work.
changed the title [-]Remove DELETE and UPDATE pipelined API options, only admin SQL[/-][+]Pipeline is insert-only; all mutations move to raw admin SQL[/+]on May 18, 2026
Summary
Lock the ingest pipeline to inserts only. Every mutation other than
INSERT—DELETE,UPDATE,TRUNCATE,DROP,ALTER,REPLACE, etc. — must be issued throughPOST /v1/query, which already gates on theadmin/servicerole (or a policy role withRawSQL: true).Why
Our policy engine authorizes mutations by evaluating row/column rules and JWT-claim check clauses against the payload of an operation (the columns being written). That model works cleanly for inserts; it does not work for predicate-driven mutations:
WHEREpredicate is satisfiable only for rows the caller's role is allowed to touch — no row-level filter injection on DML.WHERE 1=1(or a missingWHEREonDROP/TRUNCATE) wipes a table.Rather than ship a partial enforcement story, we narrow the pipeline to the one shape policy can authorize today (insert) and route everything else through the existing admin-only raw SQL gate. The policy contract stays honest — "pipelined endpoints are policy-enforced; raw SQL is admin-only" — and the WHERE-authorization design is deferred without blocking alpha.
Scope — what to remove / change
internal/ingest/bento.go: drop the inlineaction: "delete"branch (DELETE FROM <table> WHERE id = ?) and its associated DLQdelete-envelopeshape. Collapse the action check to "anything other thaninsert/empty → reject".internal/ingest/bento_test.go: remove the delete-action coverage (drain, DLQ routing, unsafe-table, double-ack) — unreachable once the branch is gone.docs/src/content/docs/architecture.md,api.md): drop the delete-envelope DLQ description and theWave-DLQ-Type: delete-envelopeheader note. State explicitly that the ingest pipeline is insert-only and that all other mutations (DELETE/UPDATE/TRUNCATE/DROP/ALTER/REPLACE) requirePOST /v1/queryunder an admin-equivalent role./v1/querymutation-response shape: confirm DDL/DML statements that return no rows from ClickHouse marshal toHTTP 200with[], not 500 (root cause of bug:TRUNCATEvia/v1/queryreturns HTTP 500 #118). Likely needs the handler to route mutations throughExecrather thanQuery, or to tolerate a nil/emptyColumnTypes()result.action: "delete"envelopes oningest.<table>subjects are no longer honored; the ingest pipeline accepts inserts only. UsePOST /v1/querywith an admin role for any other mutation./v1/tables/{table}/queryAST (internal/query/ast.go), named pipes (internal/pipes), structured query handler. Today these look read-only; verify before closing.Out of scope (deferred)
Closes
TRUNCATEvia/v1/queryreturns HTTP 500 #118 —TRUNCATEvia/v1/queryreturning HTTP 500 is the same class of bug (a mutation statement through the raw-SQL handler not returning a clean success shape). Since/v1/querybecomes the only sanctioned surface for non-insert mutations, fixing that response path is part of this work.