Repository navigation
fix(backend): sync AgentOrgNode and Secret ORM models with their migration columns (#9899) - #9989
Merged
Merged
Conversation
…columns (#9899 #9975) AgentOrgNode gains the 9 columns added by migrations 036/039/042 that the hire endpoint INSERTs but the model never declared: heartbeat_cron, heartbeat_enabled, last_heartbeat_at, adapter_type, adapter_config, context_mode, pre_pause_status, model, instructions_file_path. Secret gains workflow_id from migration 017 (#9975). Verified: create_all-built agent_org_nodes and secrets schemas are column-identical to migration-built ones on Postgres 16.
…gration 017 (#9975) Migration 20260324_017 documents the workflow scope value; the enum could not express it, leaving workflow_id half-wired at the model layer.
Closed
4 of 10 tasks
Contributor
✅ SSOT Configuration Compliance: Passing🎉 No hardcoded values detected that have SSOT config equivalents! |
This was referenced Jun 11, 2026
This was referenced Jun 11, 2026
mrveiss
added a commit
that referenced
this pull request
Jun 12, 2026
…sible invocation (#10001, #10026) (#10039) * test(migrations): reproduce unstamped-DB stranding — red acceptance test for baseline adoption (#10001) Phase A of #10001/#10026: native deploys built schemas via create_all while the Ansible alembic task silently failed (no -c flag + failed_when: false), leaving schema-without-stamp databases. 32/37 table-creating migrations use unguarded op.create_table, so a strict 'upgrade head' aborts at 001 with DuplicateTable. - test_raw_upgrade_on_unstamped_schema_fails: pins the stranding mechanism (green; permanent documentation of why baseline adoption must exist) - test_bootstrap_adopts_unstamped_schema_and_reaches_head: acceptance test for the Phase B baseline entrypoint — RED until #10001 lands - disposable-Postgres fixtures (AUTOBOT_MIGRATION_TEST_ADMIN_URL) + new migration_gate pytest marker Verified red-first: 1 failed (No module named migrations.baseline), 1 passed. Discovery confirmed while building fixtures: LLCBase.metadata.create_all fails on Postgres at head (#9980 enum NAMES vs value defaults), so stranded fleet schemas are UM-Base-shaped only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(migrations): baseline adoption for never-migrated databases (#10001) Phase B: 'python -m migrations.baseline' classifies the database before 'alembic upgrade head' and adopts schema-without-stamp deployments: 1. EMPTY -> no-op; upgrade runs the full chain 2. STAMPED (known revision) -> no-op 3. SCHEMA_NO_STAMP -> adoption: a. autogenerate diff vs head models empty -> stamp head b. else probe-ladder bracketing: per-revision artifacts (created tables, added columns, TIMESTAMPTZ markers) AST-extracted from the migration files themselves (self-maintaining); minimal-candidate selection so unobservable revisions re-run rather than being skipped c. ambiguous bracketing -> exit 3, REFUSE with recovery-doc pointer; never guess a stamp 4. STAMPED unknown revision -> compatibility table (empty today: #9759 changed zero IDs, #8225 kept its ID) else exit 4, refuse env.py URL resolution extracted to migrations/db_url.py (shared, unchanged behavior). Exit codes: 0 proceed, 2 error, 3 ambiguous, 4 foreign stamp. Tests (16, all green; Phase A acceptance test flipped red->green): - every state-machine branch incl. probe-ladder bracketing of a 010-era schema, dry-run, mixed-schema refusal, interleaved create_all refusal, foreign-stamp refusal - ladder self-checks: full chain coverage, literal create_table enforcement, observability allowlist (043/045/052), anchor spot-checks Bugs caught red-first by the suite: helper-function create_table calls (migration 010) invisible to upgrade()-only extraction -> bracketed one rung low; cross-metadata FK resolution broke autogenerate fast path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(migrations): close the create_all faucet — migrations become the only schema authority (#10001) Phase C: runtime metadata.create_all can no longer build schema on the alembic-managed database: - migrations/schema_bootstrap.py: ensure_create_all_allowed(dialect) — SQLite (app-owned local data files) stays allowed; anything else refuses with a pointer to migrations.baseline + the recovery doc unless AUTOBOT_DB_CREATE_ALL=true is set explicitly (dev/compose profiles only) - lifespan._init_skills_tables (the one remaining runtime create_all, skills SQLite engine) now calls the guard before create_all - static schema-authority sweep: AST scan of all backend runtime code fails on any metadata.create_all outside the conscious allowlist; proven by scratch-offender injection (trips on offender, green on clean tree) - guard unit tests for all three branches (sqlite / blocked / opt-in) + AST check that lifespan actually calls the guard Ground truth recorded: the historical Postgres faucet was already removed from runtime code — fleet DBs were stranded by past create_all paths; this phase prevents recurrence. Compose smoke unaffected (single_user = SQLite). Verified red-first: 4 failed before the guard existed, 5 passed after; sweep red-proven via scratch offender. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ansible): strict backend migration invocation — baseline + backup + no swallowed failures (#10001, #10026) Phase D, safe only now that baseline adoption (#10001 Phase B) exists. Both playbooks (setup-user-backend.yml, update-all-nodes.yml): - gate the sequence on Postgres-backed AUTOBOT_USER_MODE (single_user has no Postgres schema; previously alembic ran unconditionally and failed) - pg_dumpall backup before migrating (backup-node-data.yml pattern); loud warning when Postgres is remote and no local backup is possible - 'python -m migrations.baseline' adoption step BEFORE upgrade head - alembic invoked WITH '-c migrations/alembic.ini' (the old invocation could never find its config) and .env sourced so env.py can resolve the database URL - failed_when: false REMOVED — migration failure aborts the deploy with the captured alembic output update-all-nodes.yml additionally: - migration block moved after the .env regen (fresh credentials) and before the service restart (post-#9989 ORM needs columns at startup) - Play 2 sets any_errors_fatal: true — a failed node stops the rollout instead of continuing to the next node with a diverged fleet Contract pinned by tests/migrations/test_ansible_migration_contract.py (5 tests, proven red against the old playbooks, green after). ansible-playbook --syntax-check passes for both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(migrations): migration-gate matrix workflow + operator recovery runbook (#10026, #10002) Phase E — permanent gates: - .github/workflows/migration-gate.yml: postgres:16 service, runs the full matrix on every migrations/models/playbook change: (a) empty -> head, (b) stamped-intermediate -> head, (c) schema-without-stamp -> baseline adoption -> head, (d) head -> downgrade -1 -> head (new test, green on first run — the current chain's downgrade is healthy; this is a regression gate) plus baseline state-machine refusals, probe-ladder self-checks, the schema-authority sweep and the Ansible invocation contract. Minimal dependency install (exactly what env.py imports) keeps the gate fast and prevents silent dependency growth in migrations. - docs/operations/migration-recovery.md: how to read your revision, how adoption decides (state table + probe ladder), the manual re-stamp procedure for the refused/ambiguous case, foreign-stamp handling. None of the five #9759 failure classes was CI-detectable before this gate. Full local run: 27 passed against disposable Postgres 16. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(migrations): mission report — baseline adoption + strict invocation (#10001, #10026) State machine diagram, probe-ladder contents, 12 fleet states tested, red-first evidence per phase, ambiguity go/no-go assessment, and the explicit ordering statement: #10026 strictness landed only after #10001 adoption logic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(session): handoff for issue-10001 branch (#10001) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. Co-Authored-By: Paperclip <noreply@paperclip.ing> * fix(migrations): post-review hardening — CI dep gap + ansible var shadowing (#10001) Self-review findings on PR #10039: - requirements-ci had no alembic, so ci.yml's repo-wide pytest (runs on push to Dev_new_gui) would fail the probe-ladder self-checks post-merge. Added alembic==1.18.4 to requirements-ci/storage.txt AND pytest.importorskip('alembic') in the self-check module so any environment without alembic skips instead of failing. - registered ansible var 'backend_user_mode' shadowed the identically named template variable consumed by backend.env.j2 (render currently happens before the register, but the landmine is real) — renamed the register to 'backend_user_mode_probe' in both playbooks. 27/27 tests green; both playbooks pass ansible-playbook --syntax-check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(session): handoff — link follow-up issues #10044/#10045/#10046 (#10001) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #9899. Fixes #9975. Part of umbrella #9920.
Thinking Path
The hire endpoint INSERTs columns that exist only via migrations, so any schema built from ORM metadata (tests, create_all bootstraps) lacked them. Auditing the actual migration set showed the gap is 9 columns, not the 6 listed in the issue (also
context_mode,last_heartbeat_at,pre_pause_status), plus the same defect onSecret.workflow_id(filed and fixed as #9975).What Changed
models/agent_org.py: addedheartbeat_cron,heartbeat_enabled,last_heartbeat_at,adapter_type,adapter_config,context_mode(migration 20260523_036),pre_pause_status(20260525_039),model,instructions_file_path(20260525_042) — types/nullability/server defaults matching the migrations exactlymodels/secret.py: addedworkflow_id(migration 20260324_017),SecretScope.WORKFLOW, and synced the scope column commentVerification
agent_org_nodesandsecretsboth identical (name/type/nullability)modelattribute name verified safe (no SQLAlchemy/consumer collision)Merge ordering: merge after #9988 (the ORM now selects these columns on every query, so deployments must be at migration head — which #9988's startup migration step guarantees for compose).
Model Used
Fable 5 (claude-fable-5)
🤖 Generated with Claude Code