feat(profile-distribution): full approx_all pipeline (DRC-3390 PR 2)

[danyelf]

Summary

PR 2 of the DRC-3390 paired-histograms GA productionization. Replaces the PR 1 stub of ProfileDistributionTask with the full six-query approx_all orchestration: batched HLL probe + APPROX_PERCENTILE per env (continuous) + APPROX_TOP_K per env (categorical), composed on top of PR 1's per-dialect SQL renderers and capability table.

Stacks on #1389. Rebase onto main after #1389 merges.

Linear: DRC-3390

What this ships

• Strategy router pick_strategy(capabilities) → 'approx_all' | 'percentile_only' | 'unsupported', driven by recce.adapter.capabilities.
• Phase orchestration:
  • Batched HLL + min/max probe per env (one SELECT per env).
  • Batched APPROX_PERCENTILE per env (one SELECT per env).
  • Batched APPROX_TOP_K per env (one SELECT per env).
• cap_degenerate skip: columns with HLL ≥ 0.95 × row_count get an empty top-K slot (effectively unique).
• Per-column failure isolation: if a batched SELECT fails, fall back to per-column retries so one bad column doesn't blank the rest.
• Per-adapter top-K result-shape post-processor (parse_topk_result): Snowflake [[value, count], …] pairs vs DuckDB / ClickHouse flat lists vs BigQuery / Databricks STRUCTs vs Trino MAP — each normalized to (values, counts | None).
• In-memory memoization keyed by (model_unique_id, manifest_hash, version_token) on the active RecceContext. No file-backed persistence at GA per spec.
• PostHog telemetry via log_performance("profile_distribution", {strategy, total_wall_ms, phase_wall_ms, column_count, error_count, cache_hit}).

Prototype-bug fixes baked in

• DRC-3504 (timestamp epoch cast): render_epoch_cast applies a per-adapter in-SQL conversion (DuckDB epoch(), Snowflake date_part(epoch_second, …), BigQuery UNIX_SECONDS(…), etc.) before the percentile fragment. TIMESTAMP histograms render instead of silently emitting empty charts.
• DRC-3507 (DuckDB rollback): _execute_with_rollback issues an explicit ROLLBACK on any per-statement failure so DuckDB's transaction-cascade behaviour no longer blanks subsequent columns.

Payload schemas (frozen contract with PR 3)

• Continuous: {kind: "histogram", bin_edges (12), base_density (11), current_density (11), base_total, current_total} — quantile-binned, constant-area.
• Categorical: {kind: "topk", values, base_counts, current_counts, trimmed} — counts is None when the adapter's sketch doesn't expose them (DuckDB, ClickHouse).
• Unsupported tier: single envelope {status: "unsupported", reason, columns: {}}.
• Per-column failure: {kind: null}.

Test plan

• Unit tests for pick_strategy (full / percentile-only / unsupported).
• Unit tests for classify_column_type across numeric / datetime / categorical / skip.
• Unit tests for render_epoch_cast per adapter (DRC-3504 fix snapshots).
• Unit tests for parse_topk_result against every adapter shape (Snowflake pairs, DuckDB flat, ClickHouse flat, BigQuery struct, Databricks struct, Trino map).
• DuckDB integration: continuous numeric column → histogram payload with correct schema.
• DuckDB integration: low-cardinality categorical → topk payload.
• DuckDB integration: high-cardinality categorical → topk trimmed.
• DuckDB integration: UUID-cap-degenerate column → empty top-K slot.
• DuckDB integration: timestamp column (DRC-3504 regression) — emits non-empty histogram.
• DuckDB integration: deliberately-bad column (DRC-3507 regression) — other columns still succeed.
• Memoization hit/miss tests.
• Unsupported-tier short-circuit returns single envelope.
• All 45 new tests green; 278 across tests/tasks + tests/adapter pass.
• black ./recce ./tests + isort ./recce ./tests clean; flake8 clean.
• Manual smoke against Snowflake orders (8.1s cold, 0ms cached, 9 columns: 8 histogram + 1 topk).
• Manual smoke against DuckDB orders (93ms cold, 0ms cached; DuckDB top-K returns counts=None as expected).

[danyelf]

Closing as part of restructure (Ultraplan-approved). New 4-stage vertical-slice plan replaces the original 4-PR breakdown.

This PR's content splits across:

• Stage B (DuckDB-only backend): ProfileDistributionTask orchestration, DRC-3504 epoch-cast fix, DRC-3507 rollback fix, in-memory memoization, telemetry (with docstring fix — actual routing is Amplitude not PostHog), and DuckDB-targeted tests.
• Stage D (polyglot extension): multi-adapter strategy router branches (approx_all / percentile_only), per-adapter parse_topk_result() shape switch, render_epoch_cast() dialect branches, and per-dialect snapshot tests.

Branch feature/drc-3390-pr2-backend-task retained as a code-stash source.