perf(status-history): compute day buckets in a single pass

[aithal007]

Community Contribution License Agreement

By creating this pull request, I grant the project maintainers an unlimited,
perpetual license to use, modify, and redistribute these contributions under any terms they
choose, including both the AGPLv3 and the Fossorial Commercial license terms. I
represent that I have the right to grant this license for all contributed content.

AI Disclosure

Claude Code (Anthropic) was used to write the rewrite, the tests and the benchmarks, and to fuzz the new implementation against the old one. I directed the work and verified the results myself — including catching a carry-forward bug in my own first attempt, and confirming the new tests actually fail when that bug is reintroduced.

Description

Summary

computeBuckets() in server/lib/statusHistory.ts builds one bucket per day for the status-history endpoints. For each day it rescanned the entire event list — and cloned it:

const dayEvents = events.filter(...)            // full scan
const lastBeforeDay = [...events]               // full clone of every event
    .filter((e) => e.timestamp < dayStartSec)   // full scan
    .at(-1);                                    // ...to read a single element

That's three full traversals and one full array allocation per day, with days going up to 365 (the schema's own max). The clone exists solely to read its last element. A further pair of arrays was allocated per day to build allStatuses just to call .every() on it.

Separately, getBatchedStatusHistory() grouped events per entity with events.filter(...) and lastKnownEvents.find(...) inside a loop over entities — O(entities × events).

What changed

The events already arrive ordered by timestamp ascending (the queries feeding this use .orderBy(asc(statusHistory.timestamp))), and the day windows advance monotonically. So a single forward cursor yields each day's slice and the carried-in status, with no rescanning and no per-day clone. The allStatuses check became an in-place scan with an early exit, and the batched grouping became a single pass into Maps.

No behaviour change — this is purely how the same result is computed.

Impact

This is synchronous work on the request path, so it's time the process can't serve anything else.

computeBuckets, single entity (median of repeated runs):

days	events	before	after	speedup
30	500	0.63 ms	0.12 ms	5.1×
90	500	1.49 ms	0.29 ms	5.2×
90	2,000	4.35 ms	0.13 ms	33.2×
365	2,000	7.12 ms	0.45 ms	15.9×
365	5,000	17.78 ms	0.49 ms	36.4×
365	20,000	122.92 ms	1.38 ms	89.2×

Batched grouping (300 events per entity):

entities	events	before	after	speedup
10	3,000	0.29 ms	0.33 ms	0.9×
50	15,000	6.60 ms	0.20 ms	33.2×
200	60,000	108.86 ms	1.27 ms	85.4×
500	150,000	181.75 ms	2.86 ms	63.6×

Honest note: the 10-entity grouping row is a wash — building the Maps costs about what the scan did at that size. The win starts around 50 entities. The computeBuckets improvement applies at every size.

Org dashboard end-to-end (N entities × 90 days × 300 events each, computeBuckets per entity): 10 entities 9.1 ms → 4.2 ms, 50 entities 42.1 ms → 11.3 ms, 200 entities 163.4 ms → 41.9 ms.

Verification

Output equivalence — 3,006 generated workloads, zero differences. I kept a verbatim copy of the original implementation and fuzzed the two against each other over randomized combinations of days (1–90), event count (0–200), event density, priorStatus, and tzOffsetMinutes (including -720/+840 extremes), comparing full JSON.stringify output. Plus explicit edge cases: empty events, empty events with a prior status, all-unknown events, a single event, days=1, and days=365 with sparse events.

New tests (server/lib/statusHistory.test.ts) pin the properties the cursor must preserve — most importantly that a day inherits the status of the newest event from any earlier day. That's the part a naive cursor gets wrong: my own first attempt only carried forward events from before the current day and not the current day's own trailing event. I verified the test catches it by reintroducing that bug — it fails with Expected: bad, Actual: no_data.

How to test?

1. npx tsx server/lib/statusHistory.test.ts — all assertions pass.
2. npx tsc --noEmit — clean.
3. npx eslint server/lib/statusHistory.ts — clean.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟢 Approval recommended

The optimized implementations preserve ordering and carry-forward semantics, and the added tests cover the key regression cases.

Review effort: Lite
Findings: None

What changed in this PR

Optimizes status-history bucket computation and batched entity grouping while preserving existing output behavior.

Changes:

• Replaces per-day event rescans with a forward cursor.
• Avoids temporary status arrays during unknown-status checks.
• Groups batched events and prior states in linear passes.
• Adds focused regression tests for carry-forward behavior.

File	Description
server/​lib/​statusHistory.ts	Implements cursor-based bucketing and map-based grouping.
server/​lib/​statusHistory.test.ts	Adds computeBuckets behavior tests.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.