Skip to content

Manual-deploy rollback directories are never pruned: 5.5 GB and 48 warnings on every start #2525

Description

@erikdarlingdata

Found while deploying a release candidate to a dogfood box.

What

The manual deploy procedure backs the install tree up to _rollback_manual_<stamp> before copying. Nothing ever removes them. On one box:

  • 46 directories, oldest dated 2026-07-20
  • 5.48 GB consumed
  • and the service emits Directory not part of the product's layout: C:\PerformanceMonitorDarling\_rollback_manual_... once per directory — 48 warnings on every single start

That layout warning is the #2185 install-location diagnosis doing its job: it is right that these are not part of the product. But it fires for something the deploy procedure itself created, so the signal it exists to give is buried under 46 copies of a message about our own backups.

Why it matters beyond tidiness

  1. The warning is now noise. A real layout problem — a stray DLL, a half-extracted upgrade, a file dropped in by hand — arrives as warning number 47 in a list of 46 identical ones. That is the failure mode where a guard stops guarding by being too loud rather than too quiet.
  2. Unbounded disk growth on a monitoring host. 5.5 GB of dead weight, growing by ~120 MB per deploy, on a box whose whole job is to keep collecting.
  3. Nothing tells the operator. The directories are silently correct-looking, and the only complaint is a warning that reads like a misconfiguration rather than a cleanup reminder.

Shape of a fix

Whatever the deploy path becomes, the two decisions worth making explicitly:

  • Retention. Keep the last N (2 or 3 is enough to roll back a bad deploy; the older ones can only roll back to a version nobody wants). Prune the rest at deploy time, not at service start — the service should not delete things it did not create.
  • Recognition. Have the layout check know the rollback-directory naming convention and either ignore it or report it once with a count and total size, e.g. "46 rollback backups, 5.48 GB — prune with …". That converts 46 warnings into one actionable line.

The second half matters even if retention lands, because existing boxes already carry the backlog and an upgrade will not remove it.

Also worth noting from the same deploy

The deploy script's stop-guard is correct and did its job — it refused to copy while processes were still running from the install tree, naming them. Worth keeping in mind that the processes it caught were psql.exe instances left over from an operator's own diagnostic queries, not the service: anything run against the bundled store from that directory holds the tree.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions