Skip to content

feat: low-lock constraint reapplication via two-phase NOT VALID + VALIDATE#853

Open
marcus-kempe wants to merge 3 commits into
pgpartman:developmentfrom
marcus-kempe:feat/low-lock-reapply-constraints
Open

feat: low-lock constraint reapplication via two-phase NOT VALID + VALIDATE#853
marcus-kempe wants to merge 3 commits into
pgpartman:developmentfrom
marcus-kempe:feat/low-lock-reapply-constraints

Conversation

@marcus-kempe
Copy link
Copy Markdown
Contributor

@marcus-kempe marcus-kempe commented Mar 11, 2026

Summary

When reapplying constraints on large, actively-used partition sets, the AccessExclusiveLock required by ALTER TABLE ADD CONSTRAINT blocks all concurrent operations for the duration of the constraint validation scan. On partitions with millions of rows this can block production traffic for extended periods.

This PR introduces a p_low_lock parameter to reapply_constraints_proc that applies constraints in two phases per partition:

  1. Add constraints as NOT VALID (brief AccessExclusiveLock for instant DDL only)
  2. COMMIT to release the lock
  3. Validate constraints using SHARE UPDATE EXCLUSIVE (compatible with concurrent DML)

Additional improvements

  • Two-pass column loop in apply_constraints(): when multiple constraint_cols are configured, all min/max value scans are gathered first (AccessShareLock only), then all DDL statements are executed together. This prevents one column's AccessExclusiveLock from being held during another column's potentially slow min/max scan.

  • NOT VALID validation: when apply_constraints() finds an existing partman constraint that is NOT VALID and constraint_valid is true, it now validates it using ALTER TABLE ... VALIDATE CONSTRAINT (SHARE UPDATE EXCLUSIVE) instead of skipping it.

  • p_force_not_valid parameter on apply_constraints(): allows the caller to override constraint_valid and force NOT VALID constraint creation, used internally by the p_low_lock path.

Usage

-- Single call handles everything, no manual config changes needed
CALL partman.reapply_constraints_proc(
    'myschema.mytable',
    p_drop_constraints := true,
    p_apply_constraints := true,
    p_low_lock := true
);

Background

This addresses the same class of locking issues as #780 but at a broader scope. PR #780 fixed the case where the last partition's lock was held during ANALYZE. This PR addresses:

  • Lock held during min/max scans for subsequent constraint columns within apply_constraints()
  • Lock held during the full constraint validation scan (the main source of long-duration locks)

Tested against production partition sets with ~50 monthly partitions and continuous DML traffic. Without p_low_lock, constraint reapplication regularly caused lock contention and had to be manually aborted. With p_low_lock, the same operation completed without impacting concurrent operations.

Test plan

  • Tested manually against production-scale partition sets with concurrent traffic
  • New pgtap test files: test-time-procedure-weekly-low-lock-part1.sql / part2.sql verify constraints are created and validated via p_low_lock mode
  • Existing test-time-procedure-weekly-part1/part2 tests continue to pass (default behavior unchanged)
  • Verify p_force_not_valid correctly overrides constraint_valid config
  • Verify two-pass column loop produces identical constraints to single-pass

Made with Cursor

…IDATE

When reapplying constraints on large, actively-used partition sets, the
AccessExclusiveLock required by ALTER TABLE ADD CONSTRAINT blocks all
concurrent operations for the duration of the constraint validation scan.
On partitions with millions of rows this can block production traffic for
extended periods.

This change introduces three improvements to minimize lock contention:

1. Two-pass column loop in apply_constraints(): all min/max value scans
   are gathered first (AccessShareLock only), then all DDL statements
   are executed together, preventing one column's AccessExclusiveLock
   from being held during another column's potentially slow min/max scan.

2. NOT VALID constraint validation: when an existing partman constraint
   is NOT VALID and constraint_valid is true, apply_constraints() now
   validates it using ALTER TABLE VALIDATE CONSTRAINT which only requires
   SHARE UPDATE EXCLUSIVE lock (compatible with concurrent DML).

3. p_low_lock parameter on reapply_constraints_proc(): when true, each
   partition's constraints are applied in two phases:
   a) Add constraints as NOT VALID (brief AccessExclusiveLock, instant DDL)
   b) COMMIT to release the lock
   c) Validate constraints (SHARE UPDATE EXCLUSIVE, DML-compatible)

   This replaces the need for manual constraint_valid config toggling
   and separate proc calls.

A p_force_not_valid parameter is also added to apply_constraints() to
support the two-phase approach from the procedure level.

Made-with: Cursor
- Add Dockerfile, docker-compose.yml, and run_tests.sh for running
  pgtap tests in a containerized PostgreSQL 17 environment
- Fix plan count in low-lock part2 test (16 assertions, not 17)

Made-with: Cursor
Verifies that the two-pass column loop in apply_constraints() correctly
creates and validates constraints for multiple constraint columns (col1
and col2) via p_low_lock mode. Both columns get valid constraints,
confirming the deferred DDL approach produces identical results.

Made-with: Cursor
@keithf4 keithf4 added this to the 5.6 milestone Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants