perf(memory-graph): fix O(n^2) BFS queue drain in computeClusterAssignments

[iam-saiteja]

Summary

• computeClusterAssignments (groups memory-graph nodes into visual clusters) drained its BFS frontier with Array.shift(), which is O(remaining length) per call. A single wide-frontier connected component — e.g. a "hub" memory that many others derives/updates from — made the whole traversal O(n^2).
• Replaced the shift() dequeue with an index-pointer dequeue (head++). Output is byte-identical; only the dequeue mechanism changed (3 lines of actual logic).
• Added a regression test (a hub with 200 direct relations, asserting they all land in one cluster) and two benchmark scripts.

Why this matters

computeClusterAssignments runs inside a useMemo keyed on the full documents array, so it re-executes on the whole dataset every time the memory graph loads or its data changes, on the browser main thread.

Benchmarks

• bun run scripts/bench-cluster-assignments.ts imports the live function and keeps a frozen pre-fix snapshot for comparison, plus a correctness check.
• node scripts/bench-cluster-assignments-v8.mjs is a zero-dependency, engine-independent companion. This matters because this repo's dev tooling runs on Bun (JavaScriptCore), which optimizes Array.shift() much better than V8 — the engine Chrome/Edge/most browsers (i.e. this component's actual users) run. Under Bun the gap barely shows; under V8 it doesn't disappear. Both scripts print a warning if run under the wrong engine.

Averaged over 10 independent process runs on Node/V8 (not just samples within one run, to rule out GC/JIT noise):

n	previous mean (ms)	optimized mean (ms)	speedup
5,000	6.6	6.2	1.06x
20,000	76.9	36.5	2.11x
40,000	289.9	90.7	3.20x
80,000	1530.8	248.7	6.16x
100,000	2034.6	329.3	6.18x

The old implementation is also far less consistent: standard deviation ~40% of its mean at 80k, vs. ~15% for the fix. So beyond the average speedup, this also removes a source of unpredictable multi-second freezes for users with large, densely cross-referenced memory graphs.

Test plan

• bun run test (packages/memory-graph) — 198/198 passing
• bun run check-types — clean
• biome check — clean (this Windows checkout shows repo-wide CRLF-vs-LF diffs from local core.autocrlf=true; verified that's pre-existing/unrelated by checking an untouched file)
• Correctness: old and new implementations verified to produce identical output, both automatically at the start of each benchmark script and via a dedicated unit test

[iam-saiteja]

Hi @MaheshtheDev @Dhravya, whenever you get a chance, could one of you take a look at this PR?

It's a small, self-contained perf fix: computeClusterAssignments was draining its BFS queue with Array.shift(), which makes wide-frontier clusters O(n^2). I swapped it for an index-pointer dequeue, so the output is identical and only the dequeue mechanism changes. On V8 it's about 6x faster at 80k to 100k nodes, and the old version also showed much higher run-to-run variance.

There's a regression test, and all checks pass locally (198/198 tests, types and biome clean). Happy to adjust anything. Thanks!