Goal
Per R3 research (#49): engine defines what happened; policy defines how much evidence is enough. All pedagogical thresholds move out of code into a versioned policy object. Every derived state and every claim records policyVersion so a threshold change never silently rewrites what a claim meant.
Current hard-coded thresholds to move
| Threshold |
Lives in |
Becomes |
RETENTION_DELAY_MS (24h) |
projection.js |
policy.retention.minLagMs |
| remediation on any fail/partial of taught cap |
planner.js rule 3 |
policy.remediation.minConsecutiveFailures |
| ≥2 unaided successes, ≥2 sessions |
pilot-harness.evaluateClaim |
policy.independent |
| transfer/checkpoint/delayed requirements |
pilot-harness.evaluateClaim |
policy.claim |
Contract
src/vnext/policy.js: LEARNING_POLICY_V1 (frozen), resolvePolicy(version) — unknown version → fail closed, validatePolicy — malformed → fail closed.projectLearnerState(learnerId, events, capabilities, tasks, { policy }) — retention.minLagMs replaces retentionDelayMs opt (keep opt as override for tests).planNext(..., { policy }) — remediation triggers after minConsecutiveFailures consecutive fail/partial on a taught cap.evaluateClaim(..., { policy }) — thresholds from policy.independent/policy.claim; claim output carries policyVersion.- Actual lag is always derived from event timestamps — never from session labels or
now.
Engine invariants that stay hard-coded (R3 — never config)
append-only evidence · claims reconstructible from primitives · PREEXISTING never claimable · trained context ≠ held-out transfer · aided ≠ unaided · lag from timestamps · historical milestones immutable · forged/stale revisions can't alter projection · deterministic replay · failed checkpoint re-probeable (no absorbing dead-end).
Tests
- policy v1 resolves and validates; unknown/malformed version fails closed
- retention lag configurable via policy (24h vs 48h changes
retained for same events)
- remediation threshold 2: single fail does NOT route to remediation, second consecutive fail does
- claim carries
policyVersion; two policy versions on same events give explicitly different claim results
- all existing suites still green on default v1
Research basis: issue #49 round 3 comment.
Goal
Per R3 research (#49): engine defines what happened; policy defines how much evidence is enough. All pedagogical thresholds move out of code into a versioned
policyobject. Every derived state and every claim recordspolicyVersionso a threshold change never silently rewrites what a claim meant.Current hard-coded thresholds to move
RETENTION_DELAY_MS(24h)projection.jspolicy.retention.minLagMsplanner.jsrule 3policy.remediation.minConsecutiveFailurespilot-harness.evaluateClaimpolicy.independentpilot-harness.evaluateClaimpolicy.claimContract
src/vnext/policy.js:LEARNING_POLICY_V1(frozen),resolvePolicy(version)— unknown version → fail closed,validatePolicy— malformed → fail closed.projectLearnerState(learnerId, events, capabilities, tasks, { policy })—retention.minLagMsreplacesretentionDelayMsopt (keep opt as override for tests).planNext(..., { policy })— remediation triggers afterminConsecutiveFailuresconsecutive fail/partial on a taught cap.evaluateClaim(..., { policy })— thresholds frompolicy.independent/policy.claim; claim output carriespolicyVersion.now.Engine invariants that stay hard-coded (R3 — never config)
append-only evidence · claims reconstructible from primitives · PREEXISTING never claimable · trained context ≠ held-out transfer · aided ≠ unaided · lag from timestamps · historical milestones immutable · forged/stale revisions can't alter projection · deterministic replay · failed checkpoint re-probeable (no absorbing dead-end).
Tests
retainedfor same events)policyVersion; two policy versions on same events give explicitly different claim resultsResearch basis: issue #49 round 3 comment.