Repository navigation
PostgreSQL buffer-pressure cards carry what the cache HOLDS — the latest pg_buffer_usage capture as pool fill, the ten largest resident relations, the index/heap/TOAST split and the cold share, folded into the advice, so an operator learns whether shared_buffers is one relation or a working set (#3691) - #3795
Merged
Merged
Conversation
…st pg_buffer_usage capture as pool fill, the ten largest resident relations, kind and cold shares, folded into the advice (partial #3691, lane 29)
This was referenced Sep 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What the buffer-pressure card now says
A PostgreSQL card rooted on
PG_BUFFER_CACHE_PRESSURE— the memory chainPG_BUFFER_CACHE_PRESSURE → CONFIG_PG_SHARED_BUFFERS, or the composite alone on a server whose knob is not at its default — now carries apg_buffer_compositiondrill-down and a re-frozen advice block that say what the cache HOLDS beside the counters that said it is short. The composite grades hit ratio, evictions and bgwriter halts, every one a rate saying the working set does not fit; none of them says what the working set is. The latestpg_buffer_usagecapture in the window does: pool fill, the ten largest resident relations with their share of the pool, dirty share and average usage count, the split between indexes, heaps and TOAST, the dirty share of the pool, and the share sitting in relations the clock sweep evicts first. Before this the card said "50 % of block requests left shared_buffers … raise it toward a quarter of RAM" and pointed atget_pg_buffer_usage; the operator had the rate and none of the composition.Partial for #3691 — checklist line, inside the v3 line: buffer-composition drill-down (
pg_buffer_usage).The source and the shape
One read of
pg_buffer_usage(V93, already aCollectorCatalogtarget — plumbing pinned it by name for this lane): thenewestCTE picks the latestcollection_timein the window and every row is from that capture, because residency is a LEVEL and the pool totals are consistent within one capture and not across two (DarlingPgBufferUsageReader's reasoning, copied). The totals — pool used/total,buffers_listed, dirty, heap/index/TOAST, cold — are aggregates over the capture in atotalsCTE cross-joined onto the top-$4rows, never a second read against a pool that has since moved.buffers_listedis at mostpool_buffers_usedbecause the collector drops relations under 8 buffers; the payload states both rather than pretending the tail is not there. Kind shares splitbuffers_listedbyrelkind—r/mheap,iindex,tTOAST, everything else (including the NULL kind of a buffer from another database or a shared catalog, which the collector keeps because dropping it would understate the fill) as other — and sum to 1. The cold share is buffers in relations whoseavg_usage_count≤ 1: engine-defined (the clock sweep decrements on each pass and evicts at 0, so ≤ 1 is the next victim), and a PER-RELATION average, which the prose says.Names are bounded in the read (
LEFT(relation_name, $5)+length()forrelation_name_truncated, lane 7's shape) atNAMEDATALEN − 1 = 63— a collector-written name is never cut; the bound is the payload contract, not a privacy cut.The pressure figures are reused, never recomputed
miss_share,hit_ratio_suppressed,evictions,buffers_alloc,observed_ms,cache_turnovers_per_hourcome offAnalysisFinding.RootFactMetadata(the composite roots the memory chain) and ride underpressureso the rate and the level sit side by side. The fact's per-second quotients (evictions_per_sec,buffers_alloc_per_sec) are deliberately NOT re-published here: they are computed once inPgTargetFactCollector.Buffer.csover observed time and are the fact's to publish (get_analysis_facts source=pg_buffer; the advice prose states them) — the counts and the observed span ride instead, andMeasurementContractCensusTestsrule 6 (a per-second key's value must be a reader rate or a C# quotient) is why a copy under a per-second name is the wrong shape. CI caught the first push on exactly that, and onTsqlConventionGuardTests' member walk stopping short at anis { } vexpression body — both fixed in the second commit, neither rostered. The drill-down never openspg_database_stats,pg_io_statsorpg_write_stats— pinned over the comment-stripped code and the three SQL consts.Nothing to show is a sentence, and WHICH sentence is read
pg_buffer_usagecollects hourly and needspg_buffercache. A window with no capture runs two more bounded reads:pg_extension_availability's latest state forpg_buffercacheat or before the window's end (one row per distinct state, since the collector writes per database), folded installed/outdated-anywhere → cadence, else available →pg_buffercache_available("one CREATE EXTENSION away"), else absent →pg_buffercache_absent("not on this server's menu; on a managed service a parameter-group question"); and, for the cadence arm,max(collection_time)over the server so the sentence says "the last capture was N h before the window" or "this server has no capture at all". A missing availability row is reported as the cadence arm: the pass did not read a state and does not claim one. The extension read runs ONLY when there is no capture — a capture is proof the extension worked.The advice: extended, not repeated
FactAdvice.PopulateStoryTextfreezes the card from facts alone at step 3.5, before drill-downs; so, lane 16's shape, the drill-down re-freezes throughPgTargetAdvice.WithBufferComposition(frozen, summary)— words inPgTargetAdvice.Buffer.csbeside lane 2's buffer prose, read in the Darling partial. The investigation gains the value-stated sentence (Composition: the pool is 94 % full (15,400 of 16,384 buffers, 128 MB); \orders_idx` (index, appdb) alone holds 30.5 %; of the resident buffers, indexes hold 37.6 %, heaps 48.9 %, TOAST 7.5 % …; 10.1 % of the pool is dirty; 33.8 % of resident buffers sit in relations whose average usage count is at or below 1 — the clock sweep evicts those first. Captured at … UTC — a level, not the window's counters). The remediation, when a capture was read, gains "What the capture says about sizing": a relation holding ≥ 50 % of the pool is named before the knob (what reads it —get_pg_top_queries, the shape whoseshared_blks_readdwarfs itsshared_blks_hit— and its bloat); a cold MAJORITY says a larger cache would hold pages nobody re-reads, "raising it buys little here and still pays the counter-objective named above" — lane 2's D1 counter-objective sentence is stated ONCE, pinned; a pool under 90 % full says the size is not the constraint; and on Aurora (offhit_ratio_suppressed) the composition is called the honest read where the hit ratio is stated but not graded — lane 2's wording, kept. Idempotent on theComposition:marker.next_toolsfor the pressure key gainsget_pg_top_queriesbesideget_pg_buffer_usage`.The three prose selectors (0.5 dominant share, 0.5 cold majority, 0.9 fill) are NOT bars — nothing is graded on them, no fact carries them, they choose which sentence is said and the sentence states the number either way; their comments say "chosen, not measured" (the 2026-09-19 calibration held no
pg_buffer_usagefigures). No threshold, ramp or floor is added to any scorer;PgTargetScorer.Buffer.csis untouched.Gating
Inside the shared 0.5 gate, on the pressure key ONLY. The composite scores 0 below its concerning line and ≥ 0.5 at it (
GradeArm→ApplyThresholdFormula), so "on the path" already means "fired at the incident line"; theCONFIG_PG_SHARED_BUFFERSadvisory rooting alone at 0.4 on a quiet server is deliberately not enriched — the composition of a cache nothing is short of isget_pg_buffer_usage's to browse. Pinned by the root's text.What it does NOT do
PgTargetFactCollector.Buffer.csandPgTargetScorer.Buffer.csuntouched.pg_buffercache's state when a capture exists, and does not readpg_extension_availabilitythen.cold_notesays so.relname; the sentence states what is stored.PgTargetDrillDownCollector, the composer is called only from there, and the routing pins re-prove it.Boundary extensions
Darling/PerformanceMonitor.Darling.Analysis/PgTargetDrillDownCollector.cs(+8, granted): the dispatch branch and the partial declaration.Darling/PerformanceMonitor.Darling.Service/Mcp/PgTargetToolRecommendations.cs(+2): the brief namesnext_tools: get_pg_buffer_usage, get_pg_top_queries; the first was there, the second was not.PgTargetMcpSurfaceTests(registered tool,get_pg_prefix, dedup) holds.Darling/Darling.Tests/PgTargetKnobsTests.cs(one word):PlantConfigSnapshotAsyncprivate → internal so the e2e plants lane 2's defaults snapshot instead of copying it.Tests
New
Darling/Darling.Tests/PgTargetBufferDrillDownTests.cs(11 tests, 10 pure + 1 gated live):CROSS JOIN totals, the kind and coldFILTERs,LIMIT $4,LEFT/lengthpair, the extension filter,<= $2), clock discipline and dialect on all three, every FROM/JOIN a collector table or CTE; the caps.CommandTimeout = AnalysisCommandTimeoutSeconds; every read cancellable;RootFactMetadata+ the five figure names; no counter table in code or SQL; one section; re-freeze through the composer; noDeSkew; block size by name.KindLabelpartition = the SQL's;ClassifyExtensionStatesprecedence;Sharenull over zero;IsTruncatedonly on a real cut; the fixture's shares sum to 1 with every value as arithmetic.CREATE INDEX.SerializeForStoryText/TryReadStoryText, no SQL Server nouns, no DDL, noeffective_cache_size, no posture knob.DARLING_TEST_PG,[Collection("live-postgres")],LiveStoreCleanup.RunAsync): 3,000 hits + 3,000 reads a minute over one observed hour (miss 0.5 at 100 blocks/s → 1.0), lane 2's defaults snapshot, a 20-relation capture 30 min before the window's end and an OLDER capture 3 h earlier that must not be read ⇒ the REALanalyze_server(anchored) storyPG_BUFFER_CACHE_PRESSURE → CONFIG_PG_SHARED_BUFFERScarriescaptured,pool_fill 0.9399, tentop_relationsin order with the other-database row unnamed at rank 5, kind shares summing to 1,cold_share 0.3383,pressure.miss_share 0.5with the counts and observed span, the pinned sentence once and equal tonote, the sizing paragraph, bothnext_tools; a second server with the same shortage, no capture andpg_buffercacheavailable⇒pg_buffercache_available, nullcaptured_at/pool_fill, emptytop_relations, the extension sentence in the frozen investigation, no sizing paragraph.Executed here before CI: all 10 pure pins green on first run; the census set (
PgTargetFactCollectorTests,PgTargetSharedSwitchRoutingTests,PgTargetMcpSurfaceTests,PgTargetThresholdLineageTests,PgTargetMeasuredLineageTests,PgTargetV3PlumbingTests,StoreSqlClockDisciplineTests,DocCommentHygieneTests,RepoFileAdoptionTests,LivePostgresCollectionHygieneTests,LiveCleanupConversionRatchetTests,FleetIdentifierScrubTests, lane 2'sPgTargetKnobsTests, lane 16'sPgTargetDeadlockDrillDownTests) 221/221 through a net10.0 harness inside the worktree, then 221/221 with 0 skipped against a throwawaytimescale/timescaledb:2.28.1-pg18ondesktop-linux(V135) — the new e2e AND lane 2's and lane 16's e2es green with the root dispatch changed; harness and container deleted. CI's Windows job is the first execution of everything else.Which component(s) does this affect?
How was this tested?
Builds:
PerformanceMonitor.Analysis,PerformanceMonitor.Darling.Analysis,PerformanceMonitor.Darling.Service,Darling.Tests,Lite.Tests,deprecated/Dashboard.Tests— 0 warnings, 0 errors (test projects with-p:EnableWindowsTargeting=true). Pins and the live e2e executed locally as above; PostgreSQL 18 store. No production store or fleet server was read.Checklist
dotnet build -c Debug)Partial for #3691 (v3, lane 29). CHANGELOG deliberately untouched — the coordinator splices this wave's entries.