Repository navigation
fix(taps): ignore start date for default non-timestamp keys - #3798
akashmalbari wants to merge 1 commit into
Conversation
Reviewer's guide (collapsed on small PRs)Reviewer's GuideThe PR fixes incremental bookmark initialization by ignoring the default timestamp Sequence diagram for replication bookmark initializationsequenceDiagram
participant Stream
participant StateManager
Stream->>Stream: _write_starting_replication_value(context)
alt non-timestamp key and default compare_start_date
Stream->>Stream: config.pop(start_date, None)
else timestamp key or overridden compare_start_date
Stream->>Stream: retain start_date
end
Stream->>StateManager: write_starting_replication_value(context, replication_method, replication_key, config, compare_start_date)
File-Level Changes
Assessment against linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've reviewed your changes and they look great!
Sourcery assessment
Needs a human reviewer. This changes the starting replication value written to state for non-timestamp keys, so an incorrect decision could cause records to be skipped or an unnecessarily broad sync to run. Reverting would stop the behavior, but state already written may need to be reset or the affected sync rerun to repair it.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3798 +/- ##
==========================================
- Coverage 94.47% 94.44% -0.03%
==========================================
Files 74 74
Lines 6295 6298 +3
Branches 770 771 +1
==========================================
+ Hits 5947 5948 +1
- Misses 260 261 +1
- Partials 88 89 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary
start_datefrom seeding an incremental bookmark when the replication key is not timestamp-typedcompare_start_dateto convert a non-timestamp keyFixes #1677
Validation
uv run pytest tests/core/test_streams.py tests/core/test_stream_state_manager.py(89 passed)uvx prek run --files singer_sdk/streams/core.py tests/core/test_streams.py(passed)uv run mypy singer_sdk/streams/core.py(passed)uv run ty check singer_sdk/streams/core.py(passed)Compatibility and privacy/security impact
This corrects the default behavior for non-timestamp replication keys. Connectors that intentionally derive a non-timestamp bookmark from
start_datecan preserve that behavior by overridingcompare_start_date. No credentials, tokens, records, or logging behavior are changed.Summary by Sourcery
Ignore the default start date when initializing non-timestamp replication keys unless a stream explicitly supports converting it.
Bug Fixes:
Tests: