Repository navigation
PostgreSQL wait standouts, the I/O admission floor and the ratio multiple say what the second fleet read found: measured per family, cited 2026-09-20, and every Aurora wait fact stops calling a validated bar a guess (#3691 lineage flip, round 2) - #3785
Merged
Merged
Conversation
…he wait standout bars, the I/O admission floor and the ratio multiple cite the 2026-09-20 read per family, and every Aurora wait fact now carries threshold_lineage = 1 (partial #3691) No bar value changes. The wait scorer's stamp is decided by population (Aurora deltas vs stock sampled estimate) on both its exits, since the standout bar that decides a yield is measured too. The deadlock-rate and wait-profile anomalies keep 0 and their stamps say why (unplaced multiple on 22 pairs; SQL Server heavy-tail cutoff by reference; chosen ramp spans). Sampled-wait bars gain the population-0 note (§C6).
54 of 55 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The PostgreSQL bars the second fleet calibration read (batch C, 2026-09-20) now say measured, cite that read per family, and every Aurora wait fact carries
threshold_lineage = 1— partial for #3691, the checklist line Lineage flip, round 2 (calibration C). No bar value changes. The only non-comment code in the diff is the wait scorer's stamp predicate; everything else is what the code SAYS about its numbers and the flagget_analysis_factsshows.The lie this stops telling
Round 1 (#3723) flipped what the 2026-09-19 read of LEVELS could support and was honest about the rest: the wait STANDOUT bars stayed unmeasured because that read aggregated by type; the I/O quarter-hour admission floor stayed unmeasured because §B1 distributed reads per HOUR; the ratio families' firing multiple stayed unmeasured because nobody had read hour-of-week ratios at all. On 2026-09-20 the second read (the same 50 Aurora PostgreSQL clusters) went per event, per quarter-hour, and per ratio — and those three comments are now the other lie: a standout fact at
Lock:relation0.15 is telling an operator "this bar is a judgment" when it is ≈ p99.6 of that event's buckets.What flips (values in parentheses unchanged)
WaitStandoutConcerning(0.15) /WaitStandoutCritical(1.0) —PgTargetScorer.Waits.csLock:relationmax 2.01 once;LWLock:WALWrite/IO:WALSyncnever near). Stated honestly forIO:DataFileRead: 3.7 % of its buckets ≥ 0.15 (≈ p96.3), 0.25 % ≥ 1.0 — the same band the IO rollup's own 0.40 ≈ p99 occupies; reading is what a database doesIoBaselineBucketMinimumReads(250) —PgTargetScorer.Io.csbucket_reads_floormetadata still say soAnomalyThresholds.PgRatioAnomalyThreshold(3.0)PgWaitProfileFallbackMsPerSec(500 = 8× the fleet p99 0.062 ms/ms) and #3780's peak-AND-mean gate. Deadlocks/h: only 22 of 8,400 (server, bucket) pairs have a non-zero centre; no ratio ≥ 3 → an empty interval on a population too thin to place anything; theis_newarm fires in practiceThe detector flags, checked bar by bar (the brief asked me to state this)
1since PostgreSQL bars the fleet measured now say so: lineage comments cite the 2026-09-19 read and threshold_lineage flips to 1, so a validated bar stops calling itself a guess (#3691 lineage flip) #3723 — they are Z-DETECTORS onPgTpsFloor/PgTpsFallbackandPgCpuFloorPct/PgCpuFallbackPctand never grade on the ratio multiple, so §C2's TPS verdict is recorded on the constant "for the record" and flips nothing. CPU ratios were not separately measured and did not need to be: no CPU arm reads the multiple.0. Its robust arm is decided onHeavyTailModifiedZThreshold— the SQL Server fleet's cutoff by reference, unmeasured for PostgreSQL — andPgTargetScorer.ScoreRatioAnomalystamps 0 on every ratio family because its ramp spans (RatioAnomalySaturation9×,WaitProfileModifiedZSpan,FirstOccurrenceExceedanceSpan) are chosen. The stamp comment now says the multiple was READ and is routine alone, and names the floor + gate as what decides.0, as the brief directed and as the read supports — 22 pairs cannot PLACE a multiple (the flag contract needs a placed bar, not a word), and the scorer's saturation span is chosen. The stamp says exactly this.PgSampledWaitProfileFallbackMsPerSec,WaitSampledMinimumSamples): stay unmeasured, now with the note "Population 0 on the dogfood fleet as of 2026-09-20 (nopg_wait_samplingcluster)" so nobody re-reads an empty table for them (§C6).Why this shape
s_waitBarsis measured on the Aurora population and the honest verdict is the instrument's: Aurora deltas → 1, stock sampled estimate → 0. The yielded-rollup exit used to write a bare0("decided on the unmeasured standout bar"); it now writes the same population verdict, because the bar that decided it is measured. One variable, two stamps, no way for the two exits to disagree.PgRatioAnomalyThreshold's comment is a three-item list because the same number has three different verdicts, and a single adjective would be wrong for two of them.RatioAnomalySaturation. Nobody has asked where a ratio family should saturate; §C2 gives the tails (TPS p99.99 24.4, wait p99.9 31) that a future read would calibrate the ramp's top against, and the comment now says so instead of "calibrate the multiple", which is done.Falsified doc lines fixed (in-lane, smallest diff)
PgTargetAnomalyDetector.csclass remark + wait-profile summary + the classical-arm inline (unmeasured:→measured:with the two dates);PgTargetAnomalyDetector.Io.csinline "floor — unmeasured" → measured §C5;PgTargetScorer.Anomaly.csRatioAnomalySaturationsummary + the ramp's inline marker;AnomalyThresholds.cslane-9 block paragraph ("the ratio families' firing multiple is unmeasured" was about to be false). Not touched:PgTargetScorer.Queries.cs(C4 says the share bars are routine given busy — a ruling is filed on the face; lineage stays as lane L set it), the Sessions files (lane 25 is in them), any SQL, any shared switch.Proof no bar moved
The complete non-comment code diff in the two analysis assemblies:
Blast radius
Analysis-assembly comments; one predicate in the wait scorer. Every fired Aurora wait STANDOUT (
Lock:relation,LWLock:WALWrite,IO:DataFileRead,IO:WALSync) and every rollup that yields to one showsthreshold_lineage: 1on the next pass; stock sampled facts unchanged at 0. No SQL, collector, shared switch or migration; a SQL Server pass is untouched (no shared file edited).Which component(s) does this affect?
How was this tested?
Changed pins (
PgTargetMeasuredLineageTests, extended not forked):s_measuredentries now carry the DATE their block must cite (CalibrationDate2026-09-19 for round 1,SecondCalibrationDate2026-09-20 forWaitStandout*,IoBaselineBucketMinimumReads,PgRatioAnomalyThreshold); those three lefts_stillUnmeasured;RatioAnomalySaturation/WaitProfileModifiedZSpanjoined it; a news_sampledPopulationZeroroster pins the population-0 note on both sampled bars; the Waits flag pin is now "exactly twoisMeasuredPopulation ? 1 : 0stamps, no bare= 0;or= 1;".PgTargetWaitTests: standout / below-bar standout / yielded rollup0 → 1with the reason; sampled pins unchanged at 0.PgTargetWaitLiveTests: every Aurora fact1(was: only the LWLock rollup), stock still 0.PgTargetIoTests: the floor's summary must say<b>measured</b> as an admission floor, cite 2026-09-20, the 57 % figure and the hour-of-DAY consequence, and no longer sayunmeasured.PgTargetAnomalyTests: the per-detector verdict comment restated (values unchanged: TPS/CPU 1, sessions/deadlock 0).Executed locally (net10.0 harness at
obj/mactest/,AssemblyName=Darling.Tests, deleted before commit):PgTargetMeasuredLineageTests,PgTargetThresholdLineageTests,PgTargetWaitTests,PgTargetIoTests,PgTargetAnomalyTests,PgTargetSampledWaitTests,PgTargetWaitLiveTests— 93/93 green, 0 skipped, the live classes against a throwawaytimescale/timescaledb:2.28.1-pg18ondesktop-linux(torn down). Censuses:DocCommentHygieneTests,FleetIdentifierScrubTests,CommentFilterAdoptionTests,RepoFileAdoptionTests,PgTargetSharedSwitchRoutingTests,PgTargetFactCollectorTests— 120 green, 1 skipped (live).Builds (zero new warnings):
PerformanceMonitor.Analysis,PerformanceMonitor.Darling.Analysis,Darling/Darling.Tests,Lite.Tests,deprecated/Dashboard.Tests(-p:EnableWindowsTargeting=true). Pre-existing warnings untouched:ConfigChangeTraceAnchorTests.cs(94)xUnit2013,DefaultTraceEventsCollectorDefinitionTests.cs(211)xUnit2013,LogTailOverlapThresholdPinTests.cs(95)xUnit2000.CI's Windows job is the first execution of everything not in the harness set.
Checklist
dotnet build -c Debug)Partial for #3691 — checklist line: Lineage flip, round 2 (calibration C). Do not close.