Repository navigation
Staleness evaluation is invisible — say when a rule or alias was ignored, and that the check ran #347
Description
Background
A ConflictResolutionRule and a SourceAliasRule each record a snapshot of the values they were
written against. When that snapshot no longer matches the real values at staging time, the rule is
not silently reapplied — the action stages Stale instead. That is a deliberate safety mechanism
(#153): a rule encodes an assumption about data, and data drifts underneath it.
Nothing announces that this happened. ImportActionPlanner computes staleness per field
(conflictRules.TryResolve(..., out var isStale)) and per alias (sourceAliasStale), and neither the
decision nor the evaluation produces any output of its own. Two consequences:
An operator is not told their rule was ignored. A curator who writes a conflict rule and whose data
later drifts gets no signal that the rule stopped applying. The action is held for review — correct —
but nothing connects that to the rule they authored. This is the half that matters in production.
No test can confirm the mechanism ran. Found during #339's audit of the T2 suite. The seed's
per-file report: line renders a stale= count, and the audit trail records a Purge operation when
action rows are cleared, so "the reseed re-planned" and "rows were removed" are both observable. What
is not observable is whether the staleness comparison happened at all: stale=0 is produced identically
by compared the shipped rules, none had drifted and by never compared anything. A count of zero is
not evidence that something looked.
docs/automated-testing/import-and-staged-actions/16-conflict-rule-staleness.md and 17-source-alias-staleness.md
are both written against the observation this issue adds, and both fail until it exists. Per that
suite's rules a test that cannot confirm its behaviour has failed, and a failing test blocks release.
What needs to be done
- When a rule or alias is found stale, say so, naming which one and why. An operator should be able
to see that a specific rule they authored was ignored because its recorded snapshot no longer matches
the current values. This is the operator-facing half and should be visible in normal operation, not
only under raised verbosity. - When staleness is evaluated, state that it ran and over how much. Per file: how many conflict
rules were consulted, and how many source aliases. This is the half that lets a run distinguish
evaluated, none stale from never evaluated. - Zero rules considered is distinct from the check not running. A source file with no rule file at
all must produce a different observation from one whose rules were all evaluated and none drifted. - Requirement 2's statement may sit at
Debugif that is the right level for a per-file summary — a
test can raise the level, as11-batch-id-validation-and-request-log-status.mdalready does. It must
not be absent. Requirement 1 is a different judgement: an ignored rule is closer to a warning than to
a diagnostic, and normal logging should not be swallowing it. 16-conflict-rule-staleness.mdand17-source-alias-staleness.mdpass against the resulting
observations, without either constructing a drifted rule to force the mechanism to fire.
Expected tests
| Test class | Test method | Starts |
|---|---|---|
| ImportActionPlannerTests | StaleConflictRule_IsAnnouncedWithItsIdentity | ❌ |
| ImportActionPlannerTests | StaleSourceAlias_IsAnnouncedWithItsIdentity | ❌ |
| ImportActionPlannerTests | ConflictRuleEvaluation_StatesHowManyRulesWereConsulted | ❌ |
| ImportActionPlannerTests | SourceAliasEvaluation_StatesHowManyAliasesWereConsidered | ❌ |
| ImportActionPlannerTests | FileWithNoRuleFile_IsDistinguishableFromAllRulesFresh | ❌ |
| ImportActionPlannerTests | FreshRules_AreNotAnnouncedAsStale | ❌ |
Definition of done
- All expected tests listed above start red before implementation
- All requirements implemented
- All expected tests pass (green)
- No regression in related tests
- Findings summarised in a closing comment
Activity
docs [#339]: point row 25 at #347, and correct what this plan doc say…
docs [#339]: record the environment script, and tick two rows that we…
test [#411]: declare the staleness documents' dependency on #347 in t…