Skip to content

[destination-postgres] when internal table name is larger than 63 chars data get mixed up #42845

Open
@ogirardot

Description

Connector Name

destination-postgres

Connector Version

2.0.15

What step the error happened?

During the sync

Relevant information

When an original table name is already pretty long for example two tables

  • arrival_pack_arrivalpacknotification_stacklist_exclude
  • and arrival_pack_arrivalpacknotification_stacklist_include

then for both tables the underlying airbyte internal table will be truncated to :

airbyte_internal.target_cdc_raw__stream_arrival_pack_arrivalpacknotification_sta

creating corruptions and missing data as the final tables (that have names compatible with the 63 char hardcoded limit of postgresql) will be empty for one table and containing too much data for the other. (especially when both tables have the same schema)

Relevant log output

No response

Contribute

  • Yes, I want to contribute

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions