Skip to content

Port internal Postgres improvements to OSS#195

Open
rdeepak2002 wants to merge 2 commits intomainfrom
port-internal-postgres-improvements
Open

Port internal Postgres improvements to OSS#195
rdeepak2002 wants to merge 2 commits intomainfrom
port-internal-postgres-improvements

Conversation

@rdeepak2002
Copy link
Collaborator

@rdeepak2002 rdeepak2002 commented Feb 23, 2026

Pull Request type

  • Bugfix
  • Feature
  • Refactoring (no functional changes, no api changes)
  • Build related changes (Please run ./gradlew build --write-locks to refresh dependencies)
  • Other (please describe):

NOTE: Please remember to run ./gradlew spotlessApply to fix any format violations.

Changes in this PR

Ports Postgres improvements and changes from internal bdp-maestro (PR #1500, commit 7324f560f) to align the OSS database layer with internal Postgres code paths.

Key changes:

  • Replace GENERATED ALWAYS AS columns with explicit regular columns across all Flyway migrations
  • Add explicit column binding in all INSERT/UPSERT statements (no longer relying on Postgres generated columns to extract fields from JSON blobs)
  • Add SERIALIZABLE isolation level for race-condition-sensitive operations (FIRST_ONLY/LAST_ONLY run strategies, dequeue, workflow deletion)
  • Use FNV-1a hash for timeline deduplication instead of hashCode() (more robust cross-JVM consistency)
  • Add null-byte sanitization for Postgres TEXT/JSONB columns (PostgreSQL rejects \0 in text-based columns; timeline events are now sanitized via sanitizeString() before insertion)
  • Match internal SQL formatting: tuple SET syntax, ON CONFLICT column qualification, and row_to_json() without redundant ::jsonb cast
  • Rename maestro_output_data to output_data in order to match internal naming

Migration note

This is a breaking schema change. Flyway migrations are rewritten in-place (removing GENERATED columns), requiring a fresh database. Existing deployments must drop and recreate the database.

Test plan

  • All existing tests pass
  • Verified Postgres SQL query parity with internal bdp-maestro commit 7324f560f
  • Manually verified explicit column binding matches internal Postgres paths
  • Added null-byte sanitization tests ported from internal (testInsertStepInstanceSanitization, testUpdateStepInstanceSanitization)

@rdeepak2002 rdeepak2002 force-pushed the port-internal-postgres-improvements branch 2 times, most recently from 5a4dc5c to be68b1e Compare February 23, 2026 23:46
Ports Postgres improvements from internal bdp-maestro to align the OSS
database layer with internal Postgres code paths.

Key changes:
- Replace GENERATED ALWAYS AS columns with explicit regular columns
- Add explicit column binding in all INSERT/UPSERT statements
- Add SERIALIZABLE isolation level for race-condition-sensitive operations
- Use FNV-1a hash for timeline deduplication instead of hashCode()
- Use subquery pattern for DELETE...LIMIT (Postgres compatibility)
- Add null-byte sanitization for Postgres TEXT/JSONB columns
- Match internal SQL formatting: tuple SET syntax, ON CONFLICT column
  qualification, row_to_json() without redundant ::jsonb cast
- Remove CockroachDB migrations and _POSTGRES constant suffix
- Revert incorrect column renames (keep dependencies/outputs)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rdeepak2002 rdeepak2002 force-pushed the port-internal-postgres-improvements branch from ac83a25 to 13255e8 Compare February 24, 2026 04:29
@rdeepak2002 rdeepak2002 marked this pull request as ready for review February 24, 2026 04:31
Method has zero callers in OSS. The 1-param version covers all current
use cases. Addresses PR #195 review feedback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rdeepak2002 pushed a commit that referenced this pull request Feb 24, 2026
Replace boolean-flag method with explicit markTransactionSerializable()
and markTransactionSerializableReadOnly() per PR #195 review feedback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
derek-miller pushed a commit to derek-miller/maestro that referenced this pull request Mar 4, 2026
…gine] update kube-gen version 80.0.0 to 82.2.0 (Netflix#195)

* [powertrain-maestro-engine] update kube-gen version 80.0.0 to 82.2.0

* [powertrain-maestro-ui] update kube-gen version 80.0.0 to 82.2.0

* [powertrain-gateway] update kube-gen version 80.0.0 to 82.2.0

* [relay] update kube-gen version 80.0.0 to 82.2.0

---------

Co-authored-by: kube-gen-refactorator <infrastructure.compute-infra@airbnb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants