The measurement contract had ten rules and two tests: the list, numbered once, and censuses for rules 1, 2, 3, 5 and 6 (partial #3653) by erikdarlingdata · Pull Request #3701 · erikdarlingdata/PerformanceMonitor

erikdarlingdata · 2026-09-19T07:29:26Z

Partial for #3653 (item 16, "the 10-rule measurement contract as census"). Does not close the issue. Test-only: two new census files, no product code.

What does this PR do?

The contract had ten rules and two tests. #3540 stated the measurement contract in one sentence — "every delta family persists interval; readers NULL-not-0 on unknowable; rates divide by measured elapsed; epoch change → ClearServer + marker; rollups aggregate interval>0 only; per-sec names divided at comparison; gauges never delta'd" — and #3614 added the seeding rule. The review that produced the issue numbered ten; two of its numbers reached the tree (DeltaFamilyIntervalColumnTests calls the interval rule rule 4, DeltaFamilySeedingCensusTests calls the seeding rule rule 7), and the other eight stayed a sentence in an issue, un-pinned.

The list, once. Lite.Tests/MeasurementContractCensusTests' summary is the numbered list, each rule with the bound it rests on and its status (pinned by which test / pinned here / rostered / not census-able and why). The numbering is honest about what the record holds: 4 and 7 keep the numbers their tests gave them; the six remaining named rules take the free slots in the body's order (1 NULL-not-0, 2 measured elapsed, 3 epoch, 5 rollups, 6 per-second names, 8 gauges); slots 9 and 10 are left empty rather than invented — the review's enumeration was never written into the issue or its comments, and the body's order cannot be the review's (the interval rule is first in the body and fourth in the review; the epoch rule is fourth in the body and cannot be seventh). Nothing keys on a number; re-numbering is a comment edit.

What is pinned tonight, in the two existing rule tests' idiom — each fact reads source or schema, states the bound it rests on, and keeps a shrink-only roster. Every negative census carries two controls, because an empty offender list agrees with a dead regex: a positive control (a fixture the regex must fire on) and a population floor (the swept text must hold at least N of the thing the rule governs).

Rule 1 — readers NULL-not-0 on unknowable. (a) No product SQL divides by sample_interval_seconds except through NULLIF(…, 0); the regex reaches the denominator through any run of parens and function openers that are not NULLIF( (CAST(, MAX(, COALESCE(), and the control proves it fires on every such shape. (b) Every derived interval_seconds / interval_sec alias whose definition reads the stored column routes the value through NULLIF — the three-state … IS NULL → LAG test may name the column bare; the value may not. Two PgBaselineProvider arms are the named allowance (IntervalAliasPassthroughsDefendedByTheAggregatePredicate, Darling half): re-pointed by The wait and perfmon baselines stop counting every restart's fabricated zero as a quiet sample: interval-honest successor aggregates replace the legacy pair in place, the provider reads whichever supply covers its window and gates the LAG heuristic on a NULL interval, the legacy pair retires by a self-executing coverage condition, the startup backfill reads coverage off the materialization, and the Daily-routed calendar answers its unmaterialized days from raw (partial #3653: A6 + A10 mechanical half) #3698 at perfmon_interval_baseline / wait_stats_interval_baseline, whose own WHERE … IS DISTINCT FROM 0 has already dropped the interval-0 collection, so the passthrough cannot see a 0 — and the census asserts that predicate on the named source, so re-pointing either arm at raw reds the allowance. (c) Every division by the alias sits under a positive guard, and every rate arm whose CASE falls back to ELSE 0 is rostered shrink-only (RateArmsThatFallBackToZero): the two file-I/O trend columns on both SKUs, nine PostgreSQL trend rates, the two baseline wait arms. Each is dead text under its statement's own WHERE … > 0 / IS NOT NULL today; rostered because it is the shape the rule forbids, and a census that counts only live defects lets the next one arrive dead and go live in a refactor. The honest end state is END (NULL) on each — a product edit, not this lane's.
Rule 2 — rates divide by measured elapsed. Zero divisions of a delta by an assumed cadence (/ 60, / 300, / 900, / 1800, / 3600, / 86400), in SQL (delta_* dividend, no intervening slash — a / 1000.0 / 60 unit chain on a total is not a rate) and in C# (a Delta-named identifier as the dividend, so DefaultMaxGapSeconds / 60 is outside by construction). Measurement-layer campaign: the delta honesty contract (11 findings, one keystone) #3540's credits said there were none; this pins zero-and-stays-zero, floored on the 65 + 11 divisions by the measured interval that do exist.
Rule 3 — epoch change → ClearServer + marker, as a call-site census over Identity epochs: a target that restarted, failed over or was re-pointed no longer keeps subtracting from the old instance's counters — detection, forget and a logged, stored discontinuity on both hosts (partial #3653 A5) #3694's ServerEpoch: exactly the two carriers observe (CpuUtilizationCollector → ObserveInstance, PgStatementStatsCollector → ObserveStatements); every ClearServer / ClearGroups call in the product is the comparator's or a host remove path's — an exact set with counts, both directions; inside each observation the forget precedes the measure precedes the persist of the replaced pair; both hosts drain the discontinuity account after every collector run and compose the measurement onto the collection_log note (identity_epoch_changes=1 / statements_epoch_changes=1 — the A collector definition cannot contribute to collection_log.error_message, so a collector that knows why it returned nothing has nowhere to record it #3161 seam — pinned as the constants a reader keys on), and the carriers declare their state keys. Two residuals rostered shrink-only and READ off the product: FamiliesThatSubtractOnceBeforeTheCarrier = {latch_stats, procedure_stats, query_stats, spinlock_stats, wait_stats}, derived from both hosts' schedule order (CollectorScheduleDefaults.All key order for Darling, ScheduleManager.GetDefaultSchedules() for Lite) so moving the carrier's columns onto the first collector empties it; FamiliesWithoutAnEpochCarrier = {pg_wait_stats}.
Rule 5 — rollups aggregate interval > 0 only (Darling half, against the real TimescaleSupport constants by reflection, not source text): of the 22 CREATE MATERIALIZED VIEW constants, 8 read a delta family and aggregate a delta_* column. The two The wait and perfmon baselines stop counting every restart's fabricated zero as a quiet sample: interval-honest successor aggregates replace the legacy pair in place, the provider reads whichever supply covers its window and gates the LAG heuristic on a NULL interval, the legacy pair retires by a self-executing coverage condition, the startup backfill reads coverage off the materialization, and the Daily-routed calendar answers its unmaterialized days from raw (partial #3653: A6 + A10 mechanical half) #3698 successors carry sample_interval_seconds IS DISTINCT FROM 0 and are the registered members in the pair's positions; query_stats_baseline carries the equivalent delta_execution_count > 0 (bound stated: a positive execution delta is only produced from a subtraction the calculator could make); the roster RollupsThatAggregateUnknowableRows names what does not — the superseded legacy pair (perfmon_baseline, wait_stats_baseline; not registered, retiring by SupersededBaselineRelations' coverage condition) and the three hourly rollups query_stats_hourly, procedure_stats_hourly, query_stats_db_hourly. On those three a restart's (0, 0) row costs the sums nothing, but count(*) AS sample_count counts it as a sample on all three and min(delta_*) reads it as a real minimum on the two that carry a min; the daily tier is hierarchical from them and inherits both. A CAGG cannot be altered in place; the fix is The wait and perfmon baselines stop counting every restart's fabricated zero as a quiet sample: interval-honest successor aggregates replace the legacy pair in place, the provider reads whichever supply covers its window and gates the LAG heuristic on a NULL interval, the legacy pair retires by a self-executing coverage condition, the startup backfill reads coverage off the materialization, and the Daily-routed calendar answers its unmaterialized days from raw (partial #3653: A6 + A10 mechanical half) #3698's shape (successor under a new name, WITH NO DATA, backfill, coverage-gated retirement) and moves the hourly refresh phase grid — a lane, reported to the coordinator, not a line here. A second fact states why the other 14 aggregates are outside the rule (non-delta source, or hierarchical from another aggregate).
Rule 6 — per-second names divided at comparison. Every *_per_sec(ond) alias in product SQL (47 + 23) is a quotient — its select item contains a division — or a window over one (LAG(ms_per_sec)); no collector payload column carries the name; the perfmon /sec-named counters are stored as delta_cntr_value beside their interval and divided by every reader. Bound stated: the C# payload keys that carry these names copy a reader field the SQL alias populated and are not censused.
Rule 8 — gauges never delta'd: not census-able. cntr_type is not stored (Brains-review campaign: deferred structural residue (from #3538 / #3539 / #3540 / #3541) #3653 A7's rung), so nothing in the tree can say which perfmon rows are gauges. TheGaugeRule_WaitsOnTheCounterTypeRung pins the fact the wait rests on — no cntr_type column in the perfmon payload, one delta call for every counter, levels and rates both in DefaultCounters — so the rung reds the row and the census is owed. The viewer item's /sec-suffix proxy is not pinned; that lane has not landed.
Rules 4 and 7 are left to their tests; one fact keeps the header's numbers equal to the ones those files state.

Why two files. The lite CI filter reaches Lite, the shared PerformanceMonitor.* libraries (via core), the Darling Service and the Darling Viewer — and not Darling.Storage or Darling.Analysis. CrossAppGuardCiGateTests fails any Lite guard whose source read PR CI cannot run (#2839). So the Lite file sweeps exactly the ten trees it names, and Darling/Darling.Tests/MeasurementContractCensusTests sweeps the other two with the same regexes (duplicated deliberately — a cross-project equality pin would itself be a read the gate has to reach) and holds the Darling-only artifact, the continuous aggregates. It uses RepoFile.Root only (no LF reader), so RepoFileAdoptionTests needs no entry; neither file filters source lines by comment prefix, so CommentFilterAdoptionTests needs none — SQL is read through CSharpSourceWalker.StringLiteralBodies with SQL comments blanked (newlines kept), C# through StripCommentsAndStrings, so prose that quotes the forbidden shape (this repository's comments quote it often) cannot count as the shape.

What it does NOT do. No product code: the three hourly rollups, the eleven ELSE 0 arms and the two passthrough aliases are rostered and reported, not fixed. It does not recover rules 9 and 10. It does not census the C# payload keys for rule 6, the DST bucket smear (Q6), or the viewer's rendering (A7). It does not touch CHANGELOG.md, McpPageContractTests.cs, or either existing rule test.

Which component(s) does this affect?

How was this tested?

Test-only. Both projects build -c Release -p:EnableWindowsTargeting=true with zero new warnings (Lite.Tests carries dev's pre-existing xUnit2000 on LogTailOverlapThresholdPinTests.cs:95; Darling.Tests 0/0).

Every fact was executed here before push, not only compiled: the Lite half through a throwaway WPF-hosted harness (net10.0-windows, Microsoft.WindowsDesktop.App removed from the runtimeconfig) — 10/10 green, 2.2 s; the Darling half through the mactest harness (net10.0, AssemblyName=Darling.Tests) — 7/7 green, 0.3 s; the whole real Darling.Tests assembly runs on the Mac with the same 188 platform reds dev shows (WPF / SqlClient / install paths), none naming this file or the adoption censuses; CrossAppGuardCiGateTests 13/13 green against the tree with the new Lite file in it. Re-run green after rebasing onto dev at 3676970.

Mutation-checked: twelve planted defects, each restored afterwards, each turned exactly the one fact that guards it red and nothing else — a bare / sample_interval_seconds; an alias defined ELSE sample_interval_seconds; a rate arm given ELSE 0; / 60 in SQL; deltaValue / 60.0 in C#; a removed ClearServer on Lite's remove path; a per-second alias passed through raw; a PgBaselineProvider allowance re-pointed at raw; a successor CAGG stripped of its predicate (reds rule 5 AND the allowance defense); a rostered ELSE 0 retired without leaving the roster; a PostgreSQL per-second alias passed through raw; query_stats_db_hourly given the predicate without leaving the roster.

Population floors as measured tonight (each set well under the measurement, as dead-sweep detectors): Lite half — 1,089 files, 112 sample_interval_seconds mentions in SQL, 30 NULLIF(…, 0) denominators, 29 derived-alias definitions (25 stored-backed), 45 divisions by the alias, 65 deltas over a measured interval, 47 per-second aliases. Darling half — 106 files, 9 NULLIF denominators, 24 alias divisions, 11 measured, 23 per-second aliases, 22 CAGG constants, 8 governed.

Checklist

I have read the contributing guide
My code builds with zero warnings (dotnet build -c Debug)
I have tested my changes against at least one SQL Server version (test-only; no SQL Server read — the census reads source and the built assemblies)
I have not introduced any hardcoded credentials or server names

…red once, and censuses for rules 1, 2, 3, 5 and 6 (partial #3653) #3540 stated the contract in one sentence and its review numbered ten rules; two numbers reached the tree (rule 4 — every family persists its interval; rule 7 — every family seeded) and the other eight stayed a sentence in an issue. This writes the list once, in Lite.Tests/MeasurementContractCensusTests' summary, with each rule's bound and status: 4 and 7 keep the numbers their tests gave them, the six remaining named rules take the free slots in the body's order, and slots 9 and 10 are left empty rather than invented — the review's enumeration is not on the record. Pinned tonight, over source and schema, each with a positive control and a population floor so an empty offender list cannot be a dead regex: - Rule 1 (readers NULL-not-0 on unknowable): no product SQL divides by sample_interval_seconds except through NULLIF(…, 0); every derived interval_seconds alias routes the stored value through NULLIF (two PgBaselineProvider arms are the named allowance — they read a #3698 successor aggregate whose WHERE already dropped the interval-0 collection, and the census asserts that predicate on the named source); every division by the alias sits under a positive guard; every rate arm that falls back to ELSE 0 is rostered shrink-only (four file-I/O trend columns on both SKUs, nine PostgreSQL trend rates, two baseline wait arms — each dead text under its own WHERE). - Rule 2 (rates divide by measured elapsed): zero divisions of a delta by an assumed cadence, SQL and C#, both SKUs — and it stays zero. - Rule 3 (epoch → ClearServer + marker), as a call-site census over #3694's ServerEpoch: exactly the two carriers observe; every forget in the product is the comparator's or a host remove path's (exact set, both directions); both hosts drain the discontinuity account and compose the measurement onto the collection_log note; forget precedes measure precedes persist. Residuals rostered: the five families scheduled before the carrier (read off both hosts' schedule order), and pg_wait_stats, which has no carrier. - Rule 5 (rollups aggregate interval > 0 only), in the Darling half against the real TimescaleSupport constants: the two #3698 successors carry the predicate, query_stats_baseline carries the equivalent delta_execution_count > 0, and the roster names what does not — the superseded legacy pair and the three hourly rollups over query_stats / procedure_stats, whose min() and sample_count read a restart's (0, 0) row as a quiet sample. - Rule 6 (per-second names divided at comparison): every *_per_sec(ond) alias in product SQL is a quotient or a window over one; no collector payload column carries the name. - Rule 8 (gauges never delta'd) is not census-able until cntr_type is stored; the perfmon payload's lack of that column is pinned so the rung reds this row. Two files because the lite CI filter does not reach Darling.Storage or Darling.Analysis and CrossAppGuardCiGateTests fails a Lite guard PR CI cannot run: the Darling half sweeps those two trees with the same regexes and holds the continuous-aggregate census. Test-only; no product code. Every fact executed here (Lite: WPF-hosted harness, 10/10; Darling: mactest harness, 7/7; the whole Darling.Tests assembly runs with the same 188 platform reds as dev) and mutation-checked: twelve planted defects each turned exactly the one fact that guards them red.

…ned to hide it: every CASE WHEN interval > 0 THEN delta / interval now ends at END, the PostgreSQL trend readers carry the NULL instead of reading it back as 0, and both census rosters are empty (partial #3653) (#3707) #3701's measurement-contract census rostered fifteen rate arms whose CASE fell back to ELSE 0 - the file-I/O throughput trend on both SKUs, nine LAG-differenced PostgreSQL trend rates in DarlingPgTrendReader, and PgBaselineProvider's two ms_per_sec arms - as dead text under their statements' WHERE guards. Every one now ends at END (NULL: no interval, no rate), the shape #3540 rule 1 requires and the already-honest siblings write. The nine PgWaitTrendPoint / PgIoTrendPoint / PgDatabaseTrendPoint rate fields become double? and read DBNull as null; get_pg_wait_trend / get_pg_io_trend / get_pg_database_trend publish the null the way the same payloads already spell an untracked write side, and the C#-side twin evictions_per_second follows (: 0 -> : (double?)null). Two of the fifteen were not dead. The baseline arms' LAG fallback for a pre-column collection derives interval_sec = 0 when two collections date_trunc to the same second; IS NOT NULL alone let the row through and ELSE 0 rated it 0 ms/sec into the sample set (PG18 rig, three planted collections: mean 66.7 where the rated two say 100). END alone would have carried a NULL into clean and COUNT(*)ed it (count 3, mean 100), so with_rate's WHERE takes Lite's interval_sec > 0 beside IS NOT NULL, before the restart LAG (count 2, mean 100). Pinned on both texts in DarlingAnomalyBaselineTests; the nine trend arms and their nullable readers pinned in PgTrendReaderTests; both MeasurementContractCensusTests rosters empty and asserted empty.

@ServerName

…iced it had changed; Aurora's wait counters had no one watching for a restart at all: wait_stats carries the SQL Server identity first in the order, pg_wait_stats carries pg_postmaster_start_time(), and both census rosters are empty (partial #3653) (#3705) * Five families subtracted from the old instance once before anyone noticed it had changed; Aurora's wait counters had no one watching for a restart at all (partial #3653) The SQL Server identity pair (sqlserver_start_time, @@ServerName) now rides wait_stats' batch as a second result set — first in both hosts' order, observed at the end of ReadAsync and before WritePayload's first subtraction — so every SQL Server delta family re-baselines on the epoch pass instead of five of them fabricating an interval from the dead instance. The CPU carrier stays for the operator who disables wait_stats; ServerEpoch remembers per calculator which identity it last forgot to, so the second carrier does not forget the fresh baselines again. Aurora's pg_wait_stats carries pg_postmaster_start_time() as a trailing non-payload column and forgets its own two groups through the new ServerEpoch.ObservePostmaster. Both #3701 census rosters are empty and asserted empty from the product. * The Lite twin of the state-declaring roster (DefaultTraceEventsCollectorDefinitionTests) moves to six with the two new carriers

erikdarlingdata mentioned this pull request Sep 19, 2026

Brains-review campaign: deferred structural residue (from #3538 / #3539 / #3540 / #3541) #3653

Closed

39 tasks done

erikdarlingdata force-pushed the fix/3653-measurement-census branch from a0034e2 to 67936c1 Compare September 19, 2026 07:46

erikdarlingdata merged commit f08b01c into dev Sep 19, 2026
7 of 8 checks passed

erikdarlingdata deleted the fix/3653-measurement-census branch September 19, 2026 08:01

erikdarlingdata mentioned this pull request Sep 19, 2026

Changelog: second splice — 63 entries from the #3653, #3542 and #3691 waves, #3704, #3710, #3712, #3735 (65 merged PRs, two seats; 22 Added / 9 Changed / 32 Fixed; text-only) #3751

Merged

Navigation Menu

Repository navigation

The measurement contract had ten rules and two tests: the list, numbered once, and censuses for rules 1, 2, 3, 5 and 6 (partial #3653) - #3701

The measurement contract had ten rules and two tests: the list, numbered once, and censuses for rules 1, 2, 3, 5 and 6 (partial #3653)#3701
erikdarlingdata merged 1 commit into
devfrom
fix/3653-measurement-census

erikdarlingdata commented Sep 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

erikdarlingdata commented Sep 19, 2026

What does this PR do?

Which component(s) does this affect?

How was this tested?

Checklist

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant