context-graph-eval: GLiNER2 reconciliation backend, embedding-concurrency fix, #324 diagnosis

[antejavor]

Summary

Two independent pieces, both discovered/needed while diagnosing #324 for real:

1. GLiNER2 as an alternate reconciliation backend -- GLiNER2Backend (local, LLM-free extraction, unstructured2graph: pluggable extraction backends + GLiNER2 #330/unstructured2graph: extraction-quality eval, GLiNER2 vs LightRAG #332) was only reachable from unstructured2graph directly; sessions-graph/context-graph-eval hardcoded LightRAGBackend(lightrag_wrapper), so no LongMemEval question could run against a GLiNER2-built graph. SessionsGraph.reconcile_session gains an optional extraction_backend param (defaults to the original LightRAG behavior), and context_graph_eval.reconcile.reconcile_batch/the run CLI gain --extraction-backend {lightrag,gliner2}. GLiNER2 mode reconciles one session at a time (no batch/queue pipeline to fan out over -- map eval: reconciliation is sequential, and --skip-reconcile cannot actually skip it #322's pipeline is LightRAG-specific). RunMeta records it and compare() refuses across a mismatch, same as judge/agent model/tokenizer.

2. Embedding concurrency was never tuned, only the LLM concurrency was -- real run, 9/10 sessions failed with Embedding func: Worker execution timeout after 60s. _resolve_reconciliation_tuning raises MAX_PARALLEL_INSERT/MAX_ASYNC_LLM (map eval: reconciliation is sequential, and --skip-reconcile cannot actually skip it #322) for the OpenAI-backed extraction LLM, which genuinely benefits from more concurrent requests to a rate-limited remote API. LightRAG's embedding calls run through their own separate limit (EMBEDDING_FUNC_MAX_ASYNC, default 8) and timeout (EMBEDDING_TIMEOUT, default 30s -> 60s worker-kill) -- untouched by that tuning. With MAX_PARALLEL_INSERT=16 now putting many documents in flight, up to 8 fire embedding calls concurrently against the SAME local, CPU-bound bge-m3 model (context-graph-eval: cut reconciliation cost ~14x (map #297) #331) -- which contends with itself instead of parallelizing. Now lowered to 2 (serialize, which a single local model can actually sustain) and the timeout raised to 120s as headroom.

Real-world validation

Ran the actual Tier 1 LongMemEval CLI end to end multiple times against the dedicated eval instance, real OpenAI + Anthropic keys, 5 questions / --max-sessions-per-question 1 (~10 unique sessions):

• Before the embedding fix: 9/10 sessions failed reconciliation.
• After: 10/10 reconciled, both backends.

	LightRAG	GLiNER2
coverage	3/5 (60%)	2/5 (40%)
efficiency (median tokens)	45	25
reconciliation	10/10, batched	10/10, one at a time

Not a verdict on which backend is "better" -- 5 questions is a signal, not a sample size, and this is exactly the smaller-scale comparison this PR exists to make possible for the first time.

#324 diagnosis: corrected

The issue's own hypothesis was "the judge scores identical answers differently." Real testing against a frozen (--skip-reconcile) graph shows that's not quite it:

• A fixed LLMTestCase (same input/actual_output/expected_output, no retrieval involved) scored by the same judge 5x in a row: perfectly stable, both Anthropic and OpenAI (eval: the judge is hardcoded to Anthropic, but the decision was "a different provider from the pipeline" #329's provider resolution is what made testing the OpenAI side possible at all).
• But ceb54acb's retrieved answer text was byte-identical across 6 independent runs, and its covered status still flipped 0.0/0.0/1.0/0.917/0.971/0.0.
• Root cause, confirmed directly: the retrieval agent's own Cypher-query generation is non-deterministic at temperature=0 (different LIMIT/ORDER BY/column-aliasing choices call to call -- a well-documented limitation of hosted LLM APIs, not something fixable in our code). That changes retrieval_context's content and literal rendering run to run against the exact same frozen graph, which flips ContextualRecallMetric's score even when the final natural-language answer never changes.

So: the judge is stable; retrieval itself is irreducibly non-deterministic upstream of it. None of #324's judge-side mitigations (swap provider, move the 0.7 gate, median-of-N judge calls on the same input) address that -- the one that does is already on its own list: report a per-question pass rate over repeats instead of a single pass/fail, making the instability visible rather than averaging it into a headline. Not implemented in this PR -- it changes the eval loop's reporting shape, which is a design decision worth its own sign-off rather than bundling into this diff.

Test plan

• sessions-graph: 70 passed / 1 skipped (1 new)
• context-graph-eval: 193 passed / 5 skipped (6 new: extraction-backend validation, gliner2/lightrag reconciliation branching, backend-mismatch refusal in compare(), embedding-tuning knobs)
• unstructured2graph: 100 passed / 9 skipped (untouched)
• Full-workspace ty check . and ruff check/format --check: clean
• Real end-to-end: both extraction backends reconciled 10/10 real sessions after the embedding fix; the eval: the judge scores identical answers differently, which is what makes the passing set unstable #324 diagnosis above ran entirely against real, billed OpenAI + Anthropic calls

Aside, not fixed here (unrelated to this PR, filed as its own consideration): running test_e2e_reconcile.py's full file with a real key for the first time in this repo's history surfaced a pre-existing bug -- LightRAG's global shared_storage lock gets bound to one test's event loop and then reused by a later test's own loop, raising "bound to a different event loop." Reproduces in isolation from these changes.