Run the revised derivation contract: batched propose and consolidate

[antejavor]

Part of #344

Question

Does #353's derivation contract, as revised by #366, cover the corpus from any seed? #366 ran the original contract twice: one propose sample (B) matched the hand-written vocabulary, the other (A) missed a whole domain (no Location, zero answer-bearing edges), and neither proposed a relation into a core value type.

What to build (throwaway, branch only; extend derive.py on prototype/gliner2-constrained-edges)

The revised pipeline:

1. Batched propose: K (start with 4) disjoint time-stratified batches of whole sessions, each proposed independently.
2. Consolidate: one LLM call merges the batch proposals into one vocabulary, merging synonyms (Run the derivation contract once and read the vocabulary #366 saw JobRole/Occupation, Device/Possession, Food/Dish) and keeping every distinct concept. Renames are allowed here only.
3. Propose and consolidate prompts require relations into the core value types for facts stated as values, and forbid modal/tense variants (plans_to_X beside X).
4. Observe at relation_pair_cap = max_edges_per_type = 4096 (unstructured2graph: gliner2 cuts relation candidates alphabetically by type on ties #371), one turn per window.
5. Prune and validate unchanged, plus a reporting-only line for a core value type with no relation into it.
6. Stability = agreement on shared spans plus coverage (spans only one vocabulary types; answer-bearing shape), between two full derivations from different batch seeds.

Acceptance (human reading on the 10 evidence sessions, as in #366)

• Both seeds reach B's coverage: Location present, MoMA visited recovered.
• The 25:50 edge lands (personal_best-like relation into Duration).
• No tense-variant pair in either vocabulary.
• Report cost: LLM tokens (about K x $0.35 + consolidate + prune) and observe wall clock with the larger schema.

[antejavor]

Result: the revised contract fixes domain coverage for places and drops tense variants, but still loses value facts. Not accepted.

Evidence is on prototype/gliner2-constrained-edges (63c1e53): derive_batched.py, read_derived.py, derive_S{1,2}.log, read_372.log.

Two derivations came from disjoint seeds, each with 4 propose batches of 10 sessions. Both were read on the 10 evidence sessions.

Criterion	S1	S2
Place type + MoMA visited	pass: visits_place → Museum of Modern Art (0.66–0.92)	pass: visited → Museum of Modern Art (0.76–0.95)
25:50 into Duration	fail: no user→Duration relation survived prune	fail: 25:50 typed Food, giving cooks_food(user → 25:50)
No tense-variant pairs	pass	pass

	hand	S1	S2
raw / distinct edges	895 / 393	1576 / 492	1453 / 477
relations (consolidated → pruned)	19	47 → 34	51 → 38
LLM cost	—	$1.86	$1.83
observe (60 sessions, 1 core)	—	3.8 h	3.7 h

Stability: 84% agreement over 1,747 shared spans; 580 spans typed only by S1 and 498 only by S2.

Why value facts are lost: the failure moved from propose to observe/prune. Both seeds proposed relations into value types, as #366 required. But under the permissive observe pass they never fired into a value: improved_personal_best_by fired 405 times, 0 into Duration, and race_finish_time fired once, on noise. Prune dropped them, correctly given what it saw.

Second gap: S2 has no education relation at all, so it loses Business Administration, which the hand vocabulary and S1 both get. Four batches narrow #366's missing-domain failure but don't close it.

Cost: consolidate's "keep every concept" rule roughly doubles the number of relations. Observe cost scales with it, from 72 s to about 3.5 min per session.

Decision (@antejavor): observe relations into core value types with their intended endpoints, constrained into the value type. Every other relation stays permissive. Prune then sees whether each one lands on values. Follow-up: #386.