You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
QUANTILE / PERCENTILE and MODE: MEDIAN exists, the rest of the distribution does not #205
csvql has MEDIAN, which already buffers per-group values and runs a quickselect over them. MEDIAN is the 50th percentile; every other percentile is unavailable.
Verified missing on main: QUANTILE(age, 0.95), PERCENTILE_CONT(...) WITHIN GROUP (...) and MODE(age) all return error.ColumnNotFound.
Why p95 specifically
"What is the p95 latency" is one of the most common questions anyone asks of a log or metrics CSV, and it is the one number an average actively hides. A service with a 40 ms mean and a 3 s p99 looks healthy by AVG and is not. Right now csvql can give the mean, the median and the max, and nothing in between, so the interesting part of the tail is invisible.
MODE is the matching gap for categorical columns: GROUP_CONCAT gives every value and COUNT(DISTINCT) gives how many, but nothing gives the most common one without a GROUP BY ... ORDER BY COUNT(*) DESC LIMIT 1 round trip.
Implementation notes
The expensive machinery already exists. value_lists in CompactAccum buffers the values and quickselectF64 finds the k-th smallest, so QUANTILE(col, p) is the same code path with k computed from p rather than fixed at n/2, and it inherits the OptionalSlots gating from #183 for free.
Worth deciding up front:
Interpolation. DuckDB has both quantile_cont (interpolating) and quantile_disc (an actual element). csvql's MEDIAN averages the two middle values on an even count, which is the _cont behaviour, so QUANTILE should interpolate to match it.
Multiple quantiles in one query.SELECT QUANTILE(x, 0.5), QUANTILE(x, 0.95), QUANTILE(x, 0.99) should sort or select once rather than three times, since the buffer is shared.
MODE needs a different accumulator from the others: a count per distinct value, not a value list. The distinct_sets slot is a set rather than a multiset, so this is a new slot kind rather than a reuse.
MEDIAN already carries the memory cost of buffering every value in a group, so these add no new scaling problem, but they do make that cost worth documenting: it is the one aggregate whose memory is O(rows) rather than O(groups).
Activity
docs: replace the README benchmark numbers with ones that measure som…
docs: replace the README benchmark numbers with ones that measure som…