Skip to content

Explicit support + optional validation for ClickHouse JSON/object columns #236

Description

@EricAndrechek

Area: schema · ingest — FYI / feature (DX) · found via WaveHouse-Stats dogfooding (verified live on ClickHouse 26.5)

Expected: know whether a ClickHouse JSON-typed column is supported end-to-end.

Actual: it works, but only via the catch-all: JSON isn't in the explicit type switch, so a nested object falls to the default: branch (internal/discovery/validation.go:141-143 — "Unknown type — accept any value and let ClickHouse validate") and lands via JSONEachRow. Consequences: no gateway-side validation (a malformed payload fails at insert → async → DLQ, not at ingest), and system.columns shows one JSON column, so allow_columns is all-or-nothing on it (no subpath ACLs).

Impact (Stats): we moved gh_events.payload to JSON (capture-everything, deny to public, query natively by privileged roles) — relying on this catch-all behavior.

Scope: make JSON/object support explicit, optionally validate the value at the gateway, and document the all-or-nothing allow_columns caveat for JSON columns.

Related: embedded ClickHouse strict validation #175.


From WAVEHOUSE-FEEDBACK.md (WaveHouse-Stats dogfooding); validated by code-read against 0f8826c on 2026-06-04.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/docsDocumentation, site/, READMEarea/ingestIngest pipeline (Bento, batching, DLQ)documentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions