Skip to content

Staleness evaluation is invisible — say when a rule or alias was ignored, and that the check ran #347

Open

Description

@DutchJaFO

Background

A ConflictResolutionRule and a SourceAliasRule each record a snapshot of the values they were
written against. When that snapshot no longer matches the real values at staging time, the rule is
not silently reapplied — the action stages Stale instead. That is a deliberate safety mechanism
(#153): a rule encodes an assumption about data, and data drifts underneath it.

Nothing announces that this happened. ImportActionPlanner computes staleness per field
(conflictRules.TryResolve(..., out var isStale)) and per alias (sourceAliasStale), and neither the
decision nor the evaluation produces any output of its own. Two consequences:

An operator is not told their rule was ignored. A curator who writes a conflict rule and whose data
later drifts gets no signal that the rule stopped applying. The action is held for review — correct —
but nothing connects that to the rule they authored. This is the half that matters in production.

No test can confirm the mechanism ran. Found during #339's audit of the T2 suite. The seed's
per-file report: line renders a stale= count, and the audit trail records a Purge operation when
action rows are cleared, so "the reseed re-planned" and "rows were removed" are both observable. What
is not observable is whether the staleness comparison happened at all: stale=0 is produced identically
by compared the shipped rules, none had drifted and by never compared anything. A count of zero is
not evidence that something looked.

docs/automated-testing/import-and-staged-actions/16-conflict-rule-staleness.md and 17-source-alias-staleness.md
are both written against the observation this issue adds, and both fail until it exists. Per that
suite's rules a test that cannot confirm its behaviour has failed, and a failing test blocks release.

What needs to be done

  1. When a rule or alias is found stale, say so, naming which one and why. An operator should be able
    to see that a specific rule they authored was ignored because its recorded snapshot no longer matches
    the current values. This is the operator-facing half and should be visible in normal operation, not
    only under raised verbosity.
  2. When staleness is evaluated, state that it ran and over how much. Per file: how many conflict
    rules were consulted, and how many source aliases. This is the half that lets a run distinguish
    evaluated, none stale from never evaluated.
  3. Zero rules considered is distinct from the check not running. A source file with no rule file at
    all must produce a different observation from one whose rules were all evaluated and none drifted.
  4. Requirement 2's statement may sit at Debug if that is the right level for a per-file summary — a
    test can raise the level, as 11-batch-id-validation-and-request-log-status.md already does. It must
    not be absent. Requirement 1 is a different judgement: an ignored rule is closer to a warning than to
    a diagnostic, and normal logging should not be swallowing it.
  5. 16-conflict-rule-staleness.md and 17-source-alias-staleness.md pass against the resulting
    observations, without either constructing a drifted rule to force the mechanism to fire.

Expected tests

Test class Test method Starts
ImportActionPlannerTests StaleConflictRule_IsAnnouncedWithItsIdentity ❌
ImportActionPlannerTests StaleSourceAlias_IsAnnouncedWithItsIdentity ❌
ImportActionPlannerTests ConflictRuleEvaluation_StatesHowManyRulesWereConsulted ❌
ImportActionPlannerTests SourceAliasEvaluation_StatesHowManyAliasesWereConsidered ❌
ImportActionPlannerTests FileWithNoRuleFile_IsDistinguishableFromAllRulesFresh ❌
ImportActionPlannerTests FreshRules_AreNotAnnouncedAsStale ❌

Definition of done

  • All expected tests listed above start red before implementation
  • All requirements implemented
  • All expected tests pass (green)
  • No regression in related tests
  • Findings summarised in a closing comment

Activity

  1. added this to the v1.9.0 milestone on Aug 23, 2026
  2. added 2 commits that reference this issue on Aug 23, 2026
  3. added a commit that references this issue on Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions