Skip to content

Commit d3b87a5

Browse files
feat:3162 add resource name to incremental extract duplicate checks logging. (#3164)
* feat: add resource name to incremental extract duplicate check warnings. Improves debugging. Issue #3162 * fix: quote resource_name variable --------- Co-authored-by: Andrei Canache <andrei.canache@global.com>
1 parent 4c8fac8 commit d3b87a5

File tree

1 file changed

+5
-6
lines changed

1 file changed

+5
-6
lines changed

dlt/extract/incremental/__init__.py

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -646,12 +646,11 @@ def _check_duplicate_cursor_threshold(
646646
) -> None:
647647
if initial_hash_count <= Incremental.duplicate_cursor_warning_threshold < final_hash_count:
648648
logger.warning(
649-
f"Large number of records ({final_hash_count}) sharing the same value of "
650-
f"cursor field '{self.cursor_path}'. This can happen if the cursor "
651-
"field has a low resolution (e.g., only stores dates without times), "
652-
"causing many records to share the same cursor value. "
653-
"Consider using a cursor column with higher resolution to reduce "
654-
"the deduplication state size."
649+
f"Large number of records ({final_hash_count}) sharing the same value of cursor"
650+
f" field '{self.cursor_path}' on resource '{self.resource_name}'. This can happen"
651+
" if the cursor field has a low resolution (e.g., only stores dates without"
652+
" times), causing many records to share the same cursor value. Consider using a"
653+
" cursor column with higher resolution to reduce the deduplication state size."
655654
)
656655

657656

0 commit comments

Comments
 (0)