Repository navigation
No supported way to run the TTL reaper in your own account, so governed orgs can't have reaper coverage at all #625
Description
The structural half of the #613 coverage gap. #624 is "stop being silent about it"; this is "make it fixable".
The gap
The ttl-reaper Lambda is designed to run in spore.host's infra account and reach into the accounts where spores launch, by assuming a role in each (REAPER_ROLE_ARNS → spawn-ttl-reaper-ec2, lambda/ttl-reaper/README.md:85-101). Coverage therefore requires the launching account to trust an external principal.
That works when the launching account is also spore.host-owned. It cannot work when the launching account belongs to an organization with a policy against external-account trust — which is the normal state of affairs for any enterprise, university or government org, i.e. much of the intended audience.
There is no alternative path. The reaper has no CLI deploy command (ls cmd/ | grep reaper → nothing); lambda/ttl-reaper/ ships a template.yaml and is deployed by hand. resolveAccounts does already support a self-contained mode (REAPER_SCAN_SELF=true with no REAPER_ROLE_ARNS, zero cross-account trust), so the runtime is capable of this today — what's missing is any supported way to stand it up.
So for a governed org the current answer is: grant a trust their policy forbids, hand-deploy a Lambda from a template in this repo, or have no backstop. In practice they get no backstop, and per #624 nothing tells them.
This was demonstrated, not hypothesised
Attempting the documented cross-account setup into an org-governed account produced an automated security finding within 12 minutes, and the org's tooling auto-mitigated by flipping every statement in the role's trust policy from Allow to Deny. The setup is not merely discouraged there; it is actively reverted. Least-privilege didn't help — the objection was the external principal itself, not its permissions, so sts:ExternalId and tag-gated TerminateInstances conditions are irrelevant to it.
Also worth noting the existing path is awkward even where it's permitted: the README documents a chicken-and-egg first deploy (deploy the Lambda, read its role ARN, deploy the roles, wait for the next tick), a CFN stack per launching account, and CAPABILITY_NAMED_IAM — the same brittleness that led to CFN being removed from lagotto's deploy path in lagotto#154.
Ask: spawn reaper deploy, following lagotto's precedent
lagotto already solved this exact shape. lagotto deploy — "Deploy the hosted capacity poller into your own AWS account" — downloads the published Lambda artifact for a given --version, uploads it to a bucket in the user's account, and converges SNS + Lambda + EventBridge Scheduler with direct AWS API calls, no CloudFormation (lagotto#154), with --teardown, --dry-run-style gating and a doctor that verifies the result.
The reaper should get the same treatment:
spawn reaper deploystands up the reaper in the caller's own account —REAPER_SCAN_SELF=true, noREAPER_ROLE_ARNS, no cross-account trust anywhere. Intra-org and policy-compatible by construction.- Direct API calls rather than CFN, matching lagotto#154 and avoiding the per-account stack and
CAPABILITY_NAMED_IAM. --dry-runfirst, andREAPER_DRY_RUN=trueas the default on a fresh deploy — a tool whose job is terminating instances should have to be armed deliberately, not arrive enforcing.--teardown, and adoctorcheck that reports what got deployed and whether it is armed or in dry-run (feeds Nothing warns when an account has no TTL-reaper coverage, so the 'everything dies eventually' invariant silently doesn't hold #624 directly).- Keep the existing cross-account mode for the multi-account case where it is permitted; this adds a mode, it doesn't remove one.
Why this rather than a workaround
Two cheaper things were considered and rejected as answers to this:
- Deploy the reaper by hand into the affected account. Works once, for one person, and leaves the product gap in place for everyone else.
- Relocate workloads to an account the existing reaper already covers. Sound advice, and the right call for a personal account, but it's not available to an org that must launch inside its own boundary — which is the whole point of this issue.
The reaper enforces the project's core design invariant ("everything dies eventually"). Right now that invariant is only available to users who can grant spore.host cross-account access — and it's the users who can't who most need cost enforcement they don't have to remember.
Related: this belongs with the enterprise deployment work (self-hosted infra, account 812107987990). Refs #613, #624.
Activity
feat(reaper): publish the Lambda artifact, and grant the permissions …
fix(ci): tidy the lambda modules the root dependency change affected,…
reaper: spawn reaper deploy/arm/status/teardown — run the reaper in y…
chore(release): v0.115.0
scttfrdmn commentedon Oct 3, 2026
Done, in two PRs, and shipped in v0.115.0.
spawn reaper deploy/arm/status/teardownVerified on the account this was written for
The complete cycle, against the account that genuinely had no backstop:
The reap is attributed to the reaper by its own log line rather than inferred, since
sporedalso enforces TTL.And after the release, the path that could not exist until the asset did:
The released artifact ran correctly in Lambda (
fsx_scanned=3, errors=0). Account left exactly as found both times, leak-checked.Two things that differ from the issue as filed
It deploys unarmed.
REAPER_DRY_RUN=true, withspawn reaper armas a separate confirmed step. The issue proposed this and it was the right call: the schedule logs what it would reclaim against your own account's tagging before you grant it the power to terminate, which is how spawn's own reaper was rolled out. Re-deploying an armed reaper does not silently disarm it.EventBridge Rule, not Scheduler. A rule needs no separate IAM invoke role — one fewer principal for a lagotto#170-style authorization bug to hide in. It also matches what the existing CFN template creates.
A gap found on the way, worth knowing about
The reaper's execution policy granted no
fsx:orssm:actions, while its code callsfsx:DescribeFileSystems/DeleteFileSystem(the #210 ephemeral-orphan net) andssm:SendCommand/GetCommandInvocation(REAPER_GRACEFUL). In cross-account mode those run under the assumedspawn-ttl-reaper-ec2role, which does grant FSx — verified against the live role — so the gap only ever affected scan-self, which is exactly what this feature uses, and it failed silently: instances terminated correctly while the orphaned 1200 GiB filesystem #613 was reported for was left behind.pkg/reaperiamnow holds the set as data, with a test that reads the reaper's own*APIinterfaces and fails if a call has no grant — the structural end of the class behind #622 and lagotto #149/#151/#153. The CFN template's own ScanSelf policy is fixed too, so the hand-deployed path stops differing.The reverse half of that test (grant nothing uncalled) also established that
ec2:DescribeTags, which both the template and the cross-account role grant, is never called — the reaper reads tags from theDescribeInstancesresponse.Not covered here
REAPER_GRACEFUL's missing SSM grants on the cross-account role. Fixed on the Lambda's own policy here; the cross-account role still lacks them, so graceful mode silently degrades to an immediate terminate there. Worth its own issue.--artifactcovers mirrored and air-gapped setups, and is how this was tested pre-release.feat(cost): --cost-limit covers storage, not just compute (#640)
fix(lambda): package only bootstrap, not the whole source directory (#…
fix(reaper): read the live stack before deploying, so unspecified par…
fix(reaper): grant ssm: on the cross-account role so REAPER_GRACEFUL …
fix(reaper): grant ssm: on the cross-account role so REAPER_GRACEFUL …
feat(task): results_prefix — put a task's durable records where you w…
feat(reaper): report version skew between the CLI and a deployed reap…
DNS leases: expire abandoned records without credentials in the ownin…