Repository navigation
The PostgreSQL Long-Running Query read applied excludedDatabases AFTER LIMIT, so an excluded database's sessions consumed the page and the alert came back short or empty while matches existed; the list is now the candidates CTE's third flag ahead of the cap, counted on the card (Closes #3742) — and the Aurora wait-profile detector gains the peak-AND-mean gate its twins have (partial #3691) - #3780
Merged
Merged
Conversation
This was referenced Sep 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two PostgreSQL-target twins catch up with shapes the other seat landed today: the Long-Running Query read takes #3772's three-flag
candidatesCTE (closes #3742), and the Aurora wait-profile detector takes #3773's peak-AND-mean gate (partial #3691).Item 1 — the PostgreSQL LRQ read applies
excludedDatabasesin the read, ahead ofLIMIT $4(Closes #3742)The lie.
DarlingPgSessionStatesReader.GetCurrentLongRunningSessionsAsyncwasORDER BY query_duration_ms DESC LIMIT $4and then, in C#,FilterExcludedDatabasesdropped the rows whose database was on the shared list — and its own doc called that "the same known shape on both engines rather than a quiet divergence". It was the same shape: #3742 found it on all three reads and #3772 fixed the two SQL Server ones. Here it meant that on a server where an excluded reporting database's ETL held the five longest sessions, the page filled with rows the filter then threw away, and the alert came back short — or empty — while un-excluded long-running sessions existed. The list meant to make the alert accurate had switched it off for every other database, silently.The fix, #3772's shape. The evaluator's ONE builder call becomes the five-argument
LongRunningQueryExclusions.BuildSqlPredicates(programNameColumn, loginNameColumn, databaseNameColumn, excludedDatabases, firstParameterOrdinal)over the reader's three column constants — the newExclusionDatabaseNameColumn = "s.database_name"beside the knob's two — so all three arms are one text in one binding order, the database operands bound after the login operands (no ordinal that existed before this arm moves; the #3736 trap walked once). The template gains{3} AS excluded_by_databasein thecandidatesCTE, the outerWHERE NOT (program OR login OR database)ahead ofLIMIT $4, and a third scalarexcluded_by_database_counttakenAND NOT (program OR login)— the database arm is last, so a session the knob would have removed anyway is the knob's whichever database it ran in, and the three counts sum to the sessions removed (the builder's doc says why last and not first). Exact, case-insensitive — the rule the C# filter always applied — oneCOALESCE(col, '') ILIKE $n ESCAPE '\'term per normalised entry, so a row with no database name is still kept, and a blank settings entry cannot bind as''.FilterExcludedDatabasesis deleted, not disabled;BuildCurrentLongRunningSessionsSqltakes four arguments and has no three-argument sibling on purpose (a caller cannot forget the arm);LongRunningSessionsReadResultgainsExcludedByDatabase; the reader'sexcludedDatabasesparameter is gone (the list arrives as rendered text + operands, like the knob, because Storage does not reference Alerting).The receipt.
DarlingWorker.EvaluatePgLongRunningQueryAsyncrendersAlertContextBuilders.BuildLongRunningQueryExcludedDatabasesItem— the SQL Server twin's second item, same labels (Excluded By Database/Excluded Databases) — after the knob's item, only when the list is set (empty by default, so a fresh install's card is byte-identical), and then even at 0.BuildPgLongRunningQueryExclusionDetailstakes the list and the count. Annotation, never suppression: the fire is decided on the rows the read returned.Evidence, executed here (TimescaleDB 2.28.1-pg18 on
desktop-linux, torn down): the E2E plants six sessions inReportingDbabove every real one, an ETL worker in that database the knob's prefix arm also matches, sixappdbsessions and one on no database. Cap 5 with the list →[400, 304, 303, 302, 301](the no-database row and the five longest real ones),ExcludedByDatabase = 6,ExcludedByProgramPrefix = 1(the ETL worker is the knob's); without the list the same cap holds[105, 104, 103, 102, 101]— fiveReportingDbrows, the exact rows the retired shape read and threw away. Cleared knob → 7;REPORTINGDB→ the same rows (case-insensitive);Reporting→ nothing (exact, not prefix); a list naming every database →[400]alone, 13 excluded. Mutation: with the two-flagWHERErestored and the C# filter put back after the cap, the E2E failsExpected: [400, 304, 303, 302, 301] Actual: []— the page went to the excluded rows and the filter emptied it. That is the issue's lie, verbatim.Alert-path read shape: still one statement over the single latest capture, bounded by
collection_time >= $3andLIMIT $4; one more boolean column over the same rows and one more scalar over the same CTE — the SQL Server twin's cost, sub-second by shape.Item 2 — the Aurora wait-profile detector gains the peak-AND-mean gate (partial #3691)
#3691 checklist line, verbatim: "Aurora wait-profile detector lacks the peak-AND-mean gate".
Lane 9's
PgTargetAnomalyDetector.DetectWaitProfileAnomaliesgated on the window peak alone —if (modifiedZ < HeavyTailModifiedZThreshold || peakRate < PgWaitProfileFallbackMsPerSec) return;— the #3538 A8 shape #3724 removed from every other baseline detector and #3773 then removed from the SQL Server wait profile: one hot five-minute delta in a quiet window fired the profile anomaly, and a longer window bought more of them. This PR copies #3773's shape:WaitRateWindowSqlgrowsAVG(CASE WHEN interval_sec > 0 THEN … END) AS mean_ms_per_secbeside theMAX(same arm, same guard — a NULL-interval collection is NULL to both; ordinal contract 0 peak, 1 mean, 2 total, 3 samples, 4 collections); the trusted robust arm is the sharedAnomalyGate.EvaluateZScore(baseline, peakRate, meanRate, HeavyTail, HeavyTail, PgWaitProfileFallbackMsPerSec, PgWaitProfileFallbackMsPerSec, SigmaDisplayCap)PAIR call (both statistics as modified z against the bucket's median andEffectiveRobustSigma; the bar on the peak only; the same 5.0 passed twice and the family's one bar as floor and fallback because the arm'sEffectiveRobustSigma > 0guard makes the classical frame unreachable); the ratio arm asks both ratios; the no-baseline arm stays on the peak's absolute bar alone, by #3741's ruling. No bar value changes; no lineage moves — the same constants, asked of one more statistic. #3773's honest caveat (the mean's z is judged against the same per-collection median/MAD — a bounded bias toward firing, never silence) is written into the summary.Metadata stamps
mean_ms_per_sec,mean_ratio,mean_modified_zbesidecurrent_ms_per_sec/ratio/modified_z(which stay the peak's) — the sampled twin's keys (lane 24, same class), notPgAnomalyDetector'savg_ms_per_sec, because the two PostgreSQL-target profiles share one vocabulary throughPgTargetFactKeys.IsWaitProfileAnomaly.PgTargetScorer.IsExtremeWaitProfileAnomalyand the scorer readmodified_z/ratiounchanged, so the extremity escape and the grade are what they were (pinned).Evidence, executed here: live E2E with two Aurora servers, 31 days of one-minute
pg_wait_statscycling 100/200/300 ms/sec (bucket median 200, robust σ 148.26, trustworthy). QUIET window: 240 collections at the median, one at 3,200 (peak 20σ, mean 0.08σ) → noANOMALY_PG_WAIT_PROFILE. SHIFTED window: at 1,500 with the same hot collection (mean 8.8σ) → one fact,current_ms_per_sec3200,mean_ms_per_sec1507.05, both z stamped,is_new0,contrib_Lock:relationpresent, CPU excluded, nois_sampled. Red-first shown: with the gate stubbed back to peak-only, the QUIET window fires and the structural pin fails (2 red); restored, 42/42.What it does NOT do
ExcludedCountstays the knob's two-arm sum.AnomalyGate,AnomalyThresholds,FactScorer, or any bar value.is_newarm (by The wait-profile detector still fires on the window peak alone: its inline gate has no mean, unlike every detector #3724 pair-gated (Q1 residue) #3741's ruling; the sampled twin does — that divergence is stated in the comment, not hidden).PgTargetAdvice.ComposeWaitProfileRatioto name the mean (the sampled composer does; out-of-lane one-liner in the report).Tests
PgLongRunningQueryExclusionTests(extended, 10 unit): the database arm through the same builder call overs.database_nameat$7/$8after the login arm's$6, exact operands (no%, trimmed,_escaped); a list with no knob binds from$5;[]and null are the sameFALSEarm and the field text byte-for-byte; the three-flagWHEREahead ofLIMIT $4(index compare) and the thirdAND NOTscalar; a blank database arm refused; the host's exact call and the reader's new parameter pinned (and the retiredexcludedDatabases:argument absent); the receipt is the twin's second item after the knob's, only when set, even at 0, alone for a cleared knob, null-guarded; README lines. Live (PgLongRunningQueryExclusionLiveTests, 2): the The PostgreSQL Long-Running Query twin ignores the program/login opt-out knob: the setting is visible, editable and read back, and does nothing on a PostgreSQL target #3743 knob fixture now also assertsExcludedByDatabase = 0with no list; the new excluded-database E2E above.DarlingPgSessionStatesReaderTests: the stale "applied after the read" test is replaced by the third-flag pin on both renderings plus a source pin (positive control) thatFilterExcludedDatabases/excludedDatabases.Any(/ the reader's list parameter are gone. 33/33.PgTargetAnomalyTests(+2): ungated structural + arithmetic pin (ordinal contract on the outer projection; pair call regex; inline gate gone;meanRatioon the ratio arm; peak-onlyis_new; the four metadata keys; The wait-profile detector judged the window peak alone: its inline gate becomes the shared AnomalyGate pair call every other baseline detector took in #3724, so one hot collection in a quiet window stops reading as a profile shift (#3741) #3773's fixture through the detector's exactAnomalyGatecall — quiet no-fire, shifted fire, peak-only fires both; the escape reads the peak); the live two-server E2E above. 42/42 live.DarlingPgReadSqlParsesLiveTests.NotQueryFieldsgainsExclusionDatabaseNameColumnwith the census's idiom (3/3 live);LongRunningQueryExclusionsTestsdoc line that namedFilterExcludedDatabasesas "The PostgreSQL Long-Running Query twin ignores the program/login opt-out knob: the setting is visible, editable and read back, and does nothing on a PostgreSQL target #3743's" updated (36/36).RepoFileAdoptionTests,StragglerCommandTimeoutTests48,LiveCleanupConversionRatchetTests15,MeasurementContractCensusTests7 (the newAVG(CASE WHEN interval_sec > 0 …)arm carries the guarded opener),FactSourceRegistryTests,PgTargetThresholdLineageTests,StoreSqlClockDisciplineTests,FleetIdentifierScrubTests,PgTargetSharedSwitchRoutingTests27,LivePostgresCollectionHygieneTests11,DocCommentHygieneTests76/77 (the one red is the harness csproj in the tree — the documented artifact; it caught and I fixed one real stacked-summary insertion before push).-p:EnableWindowsTargeting=true— the only warnings are pre-existing on dev (ConfigChangeTraceAnchorTests.cs:94xUnit2013,LogTailOverlapThresholdPinTests.cs:95xUnit2000), none from touched files.buildis the first execution of the rest of the suite.Closes #3742. Partial #3691 — "Aurora wait-profile detector lacks the peak-AND-mean gate".
Which component(s) does this affect?
How was this tested?
See Tests — unit pins, both live classes, the mutation and the red-first executed on this machine against PostgreSQL 18 in Docker; CI's Windows job is the first execution of the rest of the suite.
Checklist
dotnet build -c Debug)