Add benchmark: small-window queries scale with total block count

[guettli]

What

Adds BenchmarkQuerySmallWindowManyBlocks, which measures a query over a small, fixed time window as the number of persisted blocks grows.

Why

A query bounded to a small time window still opens and decodes the Parquet metadata of every persisted block. In store.go, ProcessFile opens each block's data.parquet (openBlockFile → parquet.OpenFile) and only then applies the predicate to row groups in filterRowGroups — there is no step that skips a block whose time range is entirely outside the query window. So the cost of a small-window query grows with the total number of blocks stored, not with the size of the window.

The benchmark makes this visible. It writes N persisted blocks, each covering a distinct time range, then queries a window that only ever covers the newest block — the matched data is identical across runs, only the number of older, non-matching blocks changes. It reopens the store against the same bucket so the blocks are read back through the bucket-scan path rather than from memory, and asserts the query matched the newest block's rows so it can't silently regress into measuring an empty query.

Result (-benchmem -benchtime=20x), clearest in allocs/op:

blocks=1     5.1 ms    5,297 allocs/op
blocks=25    7.9 ms   12,934 allocs/op
blocks=100  13.7 ms   36,793 allocs/op
blocks=400  28.5 ms  132,202 allocs/op

Wall time grows more slowly than allocations because blocks are opened concurrently, but the total work is O(total blocks).

This is a benchmark only — it documents and measures the behaviour, it doesn't change it.

For comparison: a store that skips non-matching partitions at query time

To sanity-check that this is a property of the default backend and not of the query itself, here is the same shape (N daily partitions, query one day's window) against ClickHouse (MergeTree, PARTITION BY toYYYYMMDD(...)), read off EXPLAIN indexes=1:

partitions=1     Parts 1/1
partitions=25    Parts 1/25
partitions=100   Parts 1/100
partitions=400   Parts 1/400      (reads 1 of 400 partitions; wall time flat ~280ms)

It reads a single partition regardless of the total — the non-matching partitions are skipped at query time (they are not read, and not deleted), so cost does not grow with retained data. The absolute milliseconds are not comparable (ClickHouse is a separate server with fixed per-query overhead; FrostDB is embedded); the point is the scaling: linear vs flat.

The mechanism that enables this is per-file/partition min/max recorded outside the data file, so the filter can eliminate files before opening them. FrostDB's own Iceberg backend does exactly this (storage/iceberg.go), but the default DefaultObjstoreBucket does not.

Context

This came out of investigating slow queries on a self-hosted Parca server backed by FrostDB, where retained data (not query window) drove query latency: https://github.com/guettli/parcareport

Two notes for anyone hitting this:

• The successor storage engine, Great Lakes (per the Polar Signals / Dash0 announcement), is described as purpose-built for high-cardinality profiling data. This benchmark documents the limitation in the current default backend of FrostDB, which Parca still ships.
• ClickHouse is an open-source alternative that prunes by time as shown above, and Parca already includes an optional ClickHouse backend.

[guettlibot]

For anyone landing here from a slow single-node Parca: the downstream direction that addresses this is an embedded columnar backend.

parca-dev/parca#6357 adds an embedded DuckDB storage backend (--storage-backend=duckdb, single file / in-memory, no separate server) aimed squarely at single-node deployments. DuckDB's Parquet/columnar reader prunes by row-group time statistics — i.e. it skips blocks outside the query window, which is exactly the block-level pruning this benchmark shows the default FrostDB backend lacks.

Quick validation on a real single-node Parca (181M rows over 13.7h; CPU profile = 24M rows): the equivalent time-pruned aggregation runs in <1–2 s in DuckDB, where the FrostDB path times out on a 6 h window. A DataFusion engine is an equivalent-speed alternative (though arrow-rs is stricter and won't read FrostDB's current block Parquet — separate note).

Full landscape research + benchmarks: guettli/parcareport#31.

[guettli]

Extended: in-repo benchmark of the pruning backend (Iceberg)

The original PR compared against ClickHouse via EXPLAIN to show that a store which skips non-matching partitions stays roughly constant. Added a commit that measures this inside frostdb so the contrast is reproducible with go test and no external system.

BenchmarkQuerySmallWindowManyBlocksIceberg runs the identical workload against the Iceberg backend, sharing a harness (benchmarkSmallWindowManyBlocks) with the existing default-backend benchmark. The table is partitioned by timestamp via WithIcebergPartitionSpec, so the manifest carries per-data-file min/max bounds and the planner skips files whose bounds fall entirely outside the query window without opening them.

Result (-benchmem -benchtime=20x), allocs/op — default vs Iceberg:

blocks=1      4,624  vs   14,436
blocks=25    12,271  vs   18,284
blocks=100   36,126  vs   30,112
blocks=400  131,535  vs   77,539

From 1→400 blocks: ~28× growth (default) vs ~5× (Iceberg).

Two honest caveats, documented in the benchmark's doc comment:

• Pruning is not flat — the manifest still has one entry per data file, so reading it is O(total files); it's just far cheaper than opening and decoding each block's Parquet metadata.
• The manifest/catalog machinery adds fixed overhead, so Iceberg is more expensive at very few blocks and only wins once enough non-matching blocks accumulate (crossover ~100 blocks here).

So the benchmark now documents both the default backend's O(total blocks) cost and the mitigation already in frostdb, with numbers from the same harness.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BMDr5xaUfywx6NoNmHGgLB