Flowcus 0.10.7

v0.10.7

Features

• Resizable table columns — Drag column header borders to resize. Double-click to reset. Widths persist across sessions.
• Cache pre-heating — New settings to warm the LRU cache with bloom filters and marks from sealed hours:
  • cache_preheat_on_startup — Load index data from all sealed hours at startup (oldest-first, respects LRU budget)
  • cache_preheat_on_seal — Proactively warm cache when the merge system seals an hour

Settings

• Added Pre-heat Cache on Startup and Pre-heat Cache on Seal toggles to the Storage section in the settings panel. Both disabled by default.

Flowcus 0.10.6

v0.10.6

Frontend-only changes

Default Column Configuration

• New users get sensible defaults: Time, Src IP, Src Port, Dst IP, Dst Port, Proto, Bytes, Duration, TCP Flags
• Persisted to localStorage on first visit; "Reset" restores these defaults

Column Drag-and-Drop Reorder

• Reorder columns in the config popover via drag handles
• Reorder columns by dragging table headers directly
• Column order persisted to localStorage

Mobile & Touch Support

• Touch-based column reorder in config popover and table headers
• Touch brush selection on time histogram for zooming into time ranges
• Full-screen column config popover on small screens with close button
• Sidebar capped to viewport width, full-width on phones
• Disabled viewport zoom on input focus to prevent layout shifts

Full Changelog: v0.10.5...v0.10.6

Flowcus 0.10.5

v0.10.5

Bug Fixes

• Cursor pagination: Fix query cache returning stale page-1 results for all subsequent pages — the cache key did
  not account for after_row_id, causing infinite scroll to loop on the first page's data
• Sort tiebreaker: Default sort now uses (flowcusExportTime desc, flowcusRowId desc) so cursor boundaries are
  correct when multiple rows share the same timestamp
• Filtered pagination: Fix within-part row truncation that permanently lost rows between pages — matching indices
  are now sorted by time/rowId before truncating to the page budget

UI Changes

• Remove duplicate "out of ~X" row count from the stats bar — the histogram's "~X flows in selected range" is now
  the single source of truth for flow count estimates
• Prefix histogram flow count with ~ to indicate it is an estimate

Full Changelog: v0.10.4...v0.10.5

Flowcus 0.10.4

v0.10.4

New Tunables

• Compression Level (compression_level) — configurable zstd compression level for new parts (default 3). Lower values speed up ingestion at the cost of larger files; higher values improve compression ratio. Available in the Settings
  panel under Storage.
• Min Parts to Merge (merge_min_parts) — minimum number of parts in an hour directory before a merge is triggered (default 2). Higher values batch more parts per merge, reducing merge overhead at the cost of temporary fragmentation.
  Available under Merge & Compaction.
• Max Aggregate Rows (max_aggregate_rows) — configurable limit on rows matched during aggregation queries (default 10M). Hot-reloadable without restart. Available under Storage.

Merge Engine

• Rewritten merge engine with per-column parallelism via rayon — each column is read, gathered, encoded, and streamed directly to disk with no cross-column memory accumulation.
• Priority queue: hours with the most unmerged parts are merged first.
• Sealed hours (converged to a single part) are tracked and never re-scanned.

Observability

• New merge queue metrics exposed via Prometheus and /api/info: merge_queue_pending, merge_queue_active, merge_queue_sealed, merge_unmerged_parts, merge_columns_processed, merge_job_duration_ms.
• Health panel now shows active jobs, queue depth, unmerged parts, sealed hours, and columns processed.

Full Changelog: v0.10.3...v0.10.4

Flowcus 0.10.3

v0.10.3

Streaming Merge Engine

Redesigned the background merge system to prevent out-of-memory crashes when hour partitions accumulate thousands
of small parts exceeding available RAM.

Bounded-memory merge execution

• Merge now processes one column at a time instead of loading all columns from all source parts
  simultaneously
• Builds a global sort order by reading only the flowcusRowId column (k-way merge via min-heap), then streams
  each column through the merge order independently
• Peak memory reduced from O(all_columns × all_sources) to O(merge_order + one_column × K_sources)

Batch-limited merge plans

• Each merge batch is capped at 8 parts (configurable via merge_max_batch_size)
• Past hours no longer attempt to merge all parts in a single operation — multiple rounds converge to a single
  part via generation increments

Persistent worker threads with work queue

• Replaced per-plan spawn_blocking + semaphore with a bounded channel and long-lived worker threads
• Coordinator pushes plans to the queue; workers pull and report results back via a separate channel

Adaptive throttle controller

• Exponential ramp-down on high CPU/memory: halves available slots each cycle (4 → 2 → 1)
• Exponential ramp-up on recovery: doubles slots each cycle (1 → 2 → 4), capped at configured max
• Running merges are never interrupted — throttle only gates new dispatches
• Minimum 1 slot always available — merges never stop completely

Crash-safe .inprogress staging

• Both ingestion and merge now write to .inprogress directories invisible to the part name parser
• Atomic rename on completion makes parts visible to queries and other merges
• On startup, leftover .inprogress directories are cleaned up with a warning — source parts are always
  preserved

Configuration changes

• merge_scan_interval_secs default changed from 15 to 30
• New merge_max_batch_size option (default 8) controls maximum parts per merge batch, available in Settings
  under Merge & Compaction

Full Changelog: v0.10.2...v0.10.3

Flowcus 0.10.2

Bug Fixes

• MAC address filters — support all common notations (colons, dashes, Cisco dots, bare hex) with case-insensitive
  matching; route MAC fields through string filters instead of numeric to prevent misparse
• IPv4/IPv6 prefix field detection — use ends_with instead of contains so *PrefixLength fields (numeric U8) are
  no longer misidentified as IP address fields
• Bloom filter truncation — read column storage type and truncate lookup bytes to match actual element size,
  fixing false negatives on smaller-width columns
• MAC bloom lookups — parse MAC strings to raw 6-byte values for bloom filter point queries
• UUID7 generator — seed counter with random bits instead of zero, preventing ID collisions across independent
  generator instances observing the same millisecond

Full Changelog: v0.10.1...v0.10.2

Flowcus 0.10.1

Bug Fixes

• Structured API filters broken for non-standard IP fields — Fields like postNATSourceIPv4Address, postNATDestinationIPv4Address, and vendor-specific IP fields (Juniper, VMware, Huawei, Barracuda) were not recognized as IP
  types. Equality filters (e.g. eq "1.2.3.4") silently returned 0 rows because the address string was misinterpreted as the integer 1.
• IPv6 filter evaluation always returned 0 matches — Even for fields already recognized as IP types (sourceIPv6Address, destinationIPv6Address), the filter path read U32 from U128 columns, making all comparisons fail.
• MAC address filters never matched — String filters on MAC columns (sourceMacAddress, destinationMacAddress) read from the wrong buffer type, always producing empty values.

Improvements

• IP field detection now covers all IANA and vendor IPv4/IPv6 fields, with suffix-based fallback for unknown information elements.
• IPv6 filters support exact match, negation, CIDR, and list operations.
• MAC address string comparison is case-insensitive and supports eq/ne/in/not_in operators.
• Index pruning and bloom filter lookups extended to non-standard IP columns.

Full Changelog: v0.10.0...v0.10.1

Flowcus 0.10.0

v0.10.0

⚠️ Breaking storage format change. This release migrates parts from v1 (LZ4) to v2 (zstd + UUIDv7 row IDs). Migration runs automatically on first startup and cannot be rolled back. Back up your storage/ directory before
upgrading.

Added

• NetFlow v5 and v9 support — automatic protocol detection on the IPFIX UDP listener; v5/v9 packets are translated to IPFIX semantics internally
• Flow export — export query results as NDJSON, JSON, CSV, TSV, or LTSV from the UI
• Row detail sidebar — click a flow to inspect all fields via GET /query/row/{id}
• UUIDv7 row IDs (flowcusRowId) — stable, time-ordered unique identifier per flow; enables cursor-based pagination

Fixed

• Histogram boundary spike — records outside the query window were clamped into the first/last bucket, causing misleading spikes with relative time ranges
• Merge crash on GCD-encoded columns — zstd decompression failed with "Destination buffer is too small" due to GCD codec header not accounted for in raw_size
• Cursor pagination — switched from offset-based to keyset pagination via flowcusRowId for stable page boundaries during concurrent ingestion

Full Changelog: v0.9.4...v0.10.0

Flowcus 0.9.4

Query engine correctness

• Fixed column misalignment when reading parts with different schemas — values no longer appear under wrong column names
• Queries and merges are now coordinated with per-part read/write locks (100ms timeout) preventing mid-read deletions
• Parts are written atomically via staging directory rename — no more partially-visible parts during ingestion or merge

Memory

• Writer buffers for past partition hours are now evicted instead of accumulating indefinitely
• Merge workers invalidate storage cache entries for deleted source parts
• Query cache enforces a 128MB byte budget (was count-only)
• Aggregate queries are capped at 10M matching rows to prevent OOM

Infinite scroll

• Replaced offset-based pagination with cursor-based using flowcusExportTime boundaries
• Column schema is pinned across pages to survive background merges

Other

• Fixed picomatch ReDoS vulnerability (high severity)
• Updated all frontend and backend dependencies
• Removed and/or labels from query history display

Full Changelog: v0.9.3...v0.9.4

Flowcus 0.9.3

Previous initial release was removed:

Query results showed wrong data for columns when reading across multiple parts with different schemas. Each part's rows were built in that part's own column order, then concatenated but the final column mapping assumed all rows shared a single unified ordering. This caused column values to silently appear under wrong column names (e.g., an IP address showing up in a bytes column).

Full Changelog: https://github.com/consi/flowcus/commits/v0.9.3