Description
Connector Name
source-zendesk-support
Connector Version
4.4.0
What step the error happened?
None
Relevant information
I am running two Zendesk Support connectors in Airbyte 1.3.1 (self-hosted on AKS) syncing to a Snowflake destination. The smaller connector runs to completion with no issues. However, the second, larger connector has never successfully completed its initial sync, despite running continuously for the past 11 days. The sync has not reported failure in the UI but K8s and the log file naming shows attempt 14.
Connector version: (Zendesk Support connector version – if you know it)
Airbyte version: 1.3.1
Deployment: Self-hosted on AKS
Destination: Snowflake
The connector started syncing at a rate of about 1 million records per hour, but after a few days, it appeared to focus exclusively on the ticket_comments stream. The UI reports 24,227,780 records for ticket_comments have been synced, which matches the raw table row count in Snowflake. However, the final (i.e., deduplicated) table only shows 2,557,501 records.
A query on the raw table shows heavy duplication. Only ~250k IDs have no duplicates. Many records have upwards of 10 or even 20 duplicates.
SELECT
"_airbyte_data":id,
COUNT(*) as _count
FROM airbyte_internal."SOURCE_AIRBYTE_ZENDESK_SUPPORT_raw__stream_ticket_comments"
GROUP BY 1;
The destination state is being updated (it currently reads 2025-01-24 12:42:36.824 +0000 in the destination).
Despite multiple attempts, the sync never finishes. It’s unclear whether:
- Incremental sync is not working correctly for ticket_comments,
- The state is not being used when retried,
- Or if something in our Zendesk source is causing continuous updates to old tickets.
Happy to share logs but at this point we have several GBs of logs spread across 14 attempts. Please advise on which specific subset of logs or which logging level would be most helpful to share.
Relevant log output
Contribute
- Yes, I want to contribute