Skip to content

remove: spawn autoscale, its orchestrators, and the escalation role they used (#772) - #822

Merged
scttfrdmn merged 1 commit into
mainfrom
remove/772-autoscale
Oct 11, 2026
Merged

scttfrdmn merged 1 commit into
mainfrom
remove/772-autoscale

Conversation

@scttfrdmn

Copy link
Copy Markdown
Contributor

Closes #772

Breaking: 13 spawn autoscale subcommands removed, plus 13 exported members of
pkg/aws. Next release bumps MINOR.
~7,860 lines deleted.

The production orchestrator reconciled on rate(1 minute) from 2026-02-14 —
roughly 350,000 invocations — against a table that has never held a record.
Both groups tables measured Count=0 at deletion, so no user loses working
behaviour.

Why removal rather than tightening the role

cloudformation/autoscale-ec2-role.yaml, confirmed against the live role:

ec2:RunInstances, ec2:TerminateInstances, ec2:CreateTags  ->  *
iam:PassRole                                              ->  *
    Condition: iam:PassedToService = ec2.amazonaws.com

iam:PassedToService constrains the service, not the role. With
RunInstances on * in the same document, the holder launches an instance
carrying any EC2-assumable role in the account and reads its credentials from
IMDS. Trust was arn:aws:iam::966362334030:root — the whole infra account —
gated by an sts:ExternalId that is published in the orchestrator template.

Bounded today: no Administrator/PowerUser role in that account trusts
ec2.amazonaws.com, so there is nothing privileged to reach. It becomes serious
the moment someone creates one — which is the argument for deleting an unused
primitive rather than trusting the role inventory to stay benign. The
TerminateInstances on * was the more immediate problem: it could kill any
instance in the compute account.

Live removal

stack contents status
spawn-autoscale-orchestrator-production function, role, rule, permission, log group, table DELETE_COMPLETE
spawn-autoscale-orchestrator-staging same DELETE_COMPLETE
spawn-autoscale-ec2-role (compute account) the role DELETE_COMPLETE

Everything was CloudFormation-managed, including that role. My first instinct
was to delete the role directly; describe-stack-resources showed it belonged to
a stack, so direct deletion would have caused drift and it would have come back.
I confirmed that stack owned exactly one resource before deleting it.

Control plane re-measured with spawn footprint: 100 → 90. Not decremented
by hand — an earlier edit of that page did the arithmetic manually and was wrong.

A live route nearly went with it

handleGetCostSummary and calculateInstanceCosts lived in
lambda/dashboard-api/autoscale.go and have nothing to do with autoscale.
Deleting the file wholesale silently removed /api/cost-summary; the build
caught it.
Moved to cost_summary.go verbatim — changing cost arithmetic
inside a removal commit would hide a behaviour change in a mechanical one.

That move surfaced a finding, documented in place rather than fixed: the handler
runs on a hardcoded, undated, us-east-1-only price table that defaults an
unknown instance type to a fabricated $0.10/hr, then multiplies by 730 and
presents it as a monthly estimate. truffle#114's rule is that a fabricated price
is worse than no price. Tracked in truffle#184.

The API-surface gate earned its place

TestPublicAPISurface failed and named the 13 removed exported members and the
three consumer repos
. I verified zero consumers two ways — local checkouts of
spore-host-mcp, lagotto and calque, and an org-wide code search — before
accepting the break and running make api-snapshot.

Those symbols were added this afternoon in #774/#775, to warn when an
autoscale orchestrator ran without coverage. That question no longer exists, so
they go too.

Also resolved

  • spawn-registry-production grants removed from two IAM scripts. That table was
    already dead — dynamoRegistryTable was declared in autoscale.go and
    referenced nowhere, so the dashboard held access to a registry it never read.
  • Four stale figures fixed, all mine: CLAUDE.md's function count (stale twice
    over), the Makefile's hand-written module count (now a discovery statement —
    it had drifted 12 → 10), and two wrong claims in docs/infra-account.md.
  • nested-modules.sh reports 9 modules now, noticed by discovery with no
    code change.

Verification

make check green across all 9 nested modules, check-docs, the deploy census,
the changelog gate, semgrep (0 findings), and the full Tier 0 e2e with
-count=1 against a freshly built binary (68s).

…hey used (#772)

BREAKING: 13 `spawn autoscale` subcommands removed, plus 13 exported
members of pkg/aws. Next release bumps MINOR.

The production orchestrator reconciled on rate(1 minute) from 2026-02-14 —
roughly 350,000 invocations — against a table that has NEVER held a
record. Both groups tables measured Count=0 at deletion.

## Why removal rather than tightening the role

cloudformation/autoscale-ec2-role.yaml held, and the live role confirmed:

  ec2:RunInstances, ec2:TerminateInstances, ec2:CreateTags  ->  *
  iam:PassRole                                              ->  *
      Condition: iam:PassedToService = ec2.amazonaws.com

iam:PassedToService constrains the SERVICE, not the role. With
RunInstances on * in the same document, the holder launches an instance
carrying any EC2-assumable role in the account and reads its credentials
from IMDS. Trust was arn:aws:iam::966362334030:root — the whole infra
account — gated by an sts:ExternalId that is published in the orchestrator
template.

Bounded today: no Administrator/PowerUser role in that account trusts
ec2.amazonaws.com, so there is nothing privileged to reach. It becomes
serious the moment someone creates one — which is the argument for
deleting an unused primitive rather than trusting the role inventory to
stay benign. The TerminateInstances on * was the more immediate problem:
it could kill any instance in the compute account.

## Live removal

Three CloudFormation stacks deleted, all DELETE_COMPLETE:

  spawn-autoscale-orchestrator-production   function, role, rule,
  spawn-autoscale-orchestrator-staging      permission, log group, table
  spawn-autoscale-ec2-role  (compute acct)  the role

EVERYTHING was CloudFormation-managed, including that role. My first
instinct was to delete the role directly; describe-stack-resources showed
it belonged to a stack, so direct deletion would have caused drift and it
would have come back. I verified that stack owned exactly one resource
before deleting it.

Control plane re-measured with `spawn footprint`: 100 -> 90. Not
decremented by hand — an earlier edit of that page did the arithmetic
manually and was wrong.

## A live route nearly went with it

handleGetCostSummary and calculateInstanceCosts lived in
lambda/dashboard-api/autoscale.go and have nothing to do with autoscale.
Deleting the file wholesale silently removed /api/cost-summary; the build
caught it. Moved to cost_summary.go VERBATIM — changing cost arithmetic
inside a removal commit would hide a behaviour change in a mechanical one.

That move surfaced a finding, documented in place rather than fixed: the
handler runs on a hardcoded, UNDATED, us-east-1-only price table that
defaults an unknown instance type to a fabricated $0.10/hr, then
multiplies by 730 and shows it as a monthly estimate. truffle#114's rule
is that a fabricated price is worse than no price. Tracked in truffle#184.

## The API-surface gate earned its place

TestPublicAPISurface failed and named the 13 removed exported members and
the three consumer repos. Verified zero consumers two ways — local
checkouts of spore-host-mcp, lagotto and calque, and an org-wide code
search — before accepting the break and running `make api-snapshot`.

Those symbols were added THIS AFTERNOON in #774/#775 to warn when an
autoscale orchestrator ran without coverage. That question no longer
exists, so they go too.

## Also resolved

spawn-registry-production grants removed from two IAM scripts. That table
was already dead: dynamoRegistryTable was declared in autoscale.go and
referenced nowhere, so the dashboard held access to a registry it never
read. Found by the data-chain audit; removed here as a side effect.

Four stale figures fixed, all mine: CLAUDE.md's function count (stale
twice over), the Makefile's hand-written module count (now a discovery
statement — the count had drifted 12 -> 10), and two wrong claims in
docs/infra-account.md.

nested-modules.sh reports 9 modules now, down from 10, noticed by
discovery with no code change.

make check green, check-docs, census, changelog, semgrep 0 findings, Tier
0 e2e with -count=1 against a fresh binary (68s).
@scttfrdmn scttfrdmn added this to the Security audit 2026-10 milestone Oct 11, 2026
@codecov

codecov Bot commented Oct 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@scttfrdmn
scttfrdmn merged commit c8a699a into main Oct 11, 2026
7 checks passed
@scttfrdmn
scttfrdmn deleted the remove/772-autoscale branch October 11, 2026 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Two autoscale orchestrators have polled every minute since February against empty tables

1 participant