Skip to content

Serialize concurrent apply operations across planning and convergence checks #16

Open

Description

@ORESoftware

Audit finding

dpm apply currently plans from one snapshot, executes the emitted statements, and then re-introspects for convergence without a database-wide migration lock spanning that full window. Two deployers targeting the same database can therefore race: each may plan from stale state, interleave DDL, or misattribute residual drift.

Proposed work

  • Define a deterministic lock identity per target database/schema set.
  • Add a PostgreSQL implementation using a session-level advisory lock held across planning, apply, and final convergence verification.
  • Design a CockroachDB-safe equivalent, likely a lease row with transactional compare-and-set semantics and expiry rather than PostgreSQL advisory-lock compatibility assumptions.
  • Add configurable lock acquisition timeout and actionable diagnostics.
  • Ensure the lock is released on success, error, cancellation, and process termination where the backend permits it.
  • Add live stress tests with two concurrent dpm apply processes against PostgreSQL and CockroachDB.
  • Document behavior for read-only diff/verify commands and multi-schema targets.

Acceptance criteria

  • Concurrent applies against one target cannot execute migration statements concurrently.
  • A timed-out contender exits non-zero without changing the target.
  • PostgreSQL and CockroachDB live tests cover lock acquisition, timeout, owner failure, and subsequent recovery.

Discovered during the hardening work in #15.

Activity

  1. ORESoftware commented on Sep 15, 2026

    @ORESoftware
    CollaboratorAuthor

    2026-09-15 coordination-boundary refinement:

    The implementation target for this issue is now ORESoftware/ores-locks-and-leases, not a second DPM-owned public lease abstraction. The current PostgresMigrationLease remains a legacy compatibility mechanism while the shared package is adopted.

    Safe rollout requirement: do not swap DEFAULT_MIGRATION_LOCK_KEY directly to the shared string-derived advisory key. During a compatibility window, new executors must preserve exclusion with legacy-only executors while also acquiring the canonical shared identity declarative-migrations/migrations/apply:{target_id}. Acquire/release ordering, lease renewal immediately before irreversible boundaries, fencing-token stale-writer rejection, and cleanup-error precedence must follow the shared package contract.

    PR #71 declares the Zed runtime dependency and documents the dual-lock transition. Completion still requires resolver-produced .zpkg.lock, actual Rust runtime consumption, and exact *-test qualification for mixed-version contention, expiry/renewal, fencing, crash/restart and subsequent recovery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions