CBG-4286 separate feed name from checkpoint prefix#8073
Open
CBG-4286 separate feed name from checkpoint prefix#8073
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors DCP client initialization to separate the feed name (used for DCP stream naming) from the checkpoint prefix (used for checkpoint document naming). Previously, these were conflated, with the feed ID being appended to the checkpoint prefix within the DCP client. Now, the FeedID and CheckpointPrefix are independent fields in DCPClientOptions.
Changes:
- Introduced FeedID field in DCPClientOptions for generating unique DCP stream names
- Modified checkpoint prefix generation to include complete paths at the call site
- Converted attachment compaction phase constants to a typed constant for better type safety
- Updated all DCP client creation sites to provide explicit FeedID and CheckpointPrefix values
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| base/gocb_dcp_client.go | Added FeedID field to DCPClientOptions, removed ID parameter from NewDCPClient, generate stream name from FeedID inside the client |
| base/dcp_common.go | Added length validation for generated DCP stream names (max 200 characters) |
| base/gocb_dcp_feed.go | Removed feedName parameter from NewDCPClient call (but missing FeedID in options) |
| base/dcp_client_test.go | Updated all test calls to remove feedID parameter from NewDCPClient |
| db/attachment_compaction.go | Introduced attachmentCompactionPhase type, updated functions to use FeedID and complete checkpoint prefix, removed GenerateCompactionDCPStreamName |
| db/attachment_compaction_test.go | Updated to use new getCompactionDCPClientOptions signature (but has compilation error) |
| db/background_mgr_resync_dcp.go | Updated to use FeedID and complete checkpoint prefix, renamed GenerateResyncDCPStreamName to GetResyncDCPCheckpointPrefix |
| db/background_mgr_resync_dcp_test.go | Updated test to use new function signature |
| db/background_mgr_attachment_migration.go | Updated to use FeedID and complete checkpoint prefix, removed GenerateAttachmentMigrationDCPStreamName |
| db/background_mgr_attachment_migration_test.go | Updated test to use new function signature |
| db/background_mgr_attachment_compaction.go | Updated parameter type to attachmentCompactionPhase for better type safety |
| db/util_testing.go | Added FeedID to purge DCP client options |
| rest/attachmentcompactiontest/attachment_compaction_api_test.go | Updated to use new checkpoint prefix function (but has type mismatch issue) |
| rest/adminapitest/resync_test.go | Updated to use new checkpoint prefix function |
| tools/cache_perf_tool/dcpDataGeneration.go | Added FeedID to test DCP client options, removed feedID parameter from NewDCPClientForTest |
224ef62 to
a536888
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CBG-4286 separate feed name from checkpoint prefix
The only behavior change is that https://github.com/couchbase/sync_gateway/blob/main/base/dcp_client_worker.go#L149 will ignore the given DCP client's checkpoint prefix rather than something that is closer to
_sync:dcp_ck. I can restore this behavior but I'm not super worried about it in practice since we only persist metadata periodically anyway.Please do a careful review to see if I've accidentally changed the checkpoint prefixes, but this is why I wrote 4717efb
The feed name is only visible via cbcollect. It needs to be:
DCPAgentthat opens, or else kv will drop the connection. Hence appending UUIDOnly the code that uses
GoCBDCPClientwith meaningful checkpoints has a very small behavioral change with the code. In this case, if you were running a feed with checkpoints at the same time as another feed, it would previously ignore the other feed's checkpoint documents and a callback. In practice, the callbacks return immediately for_syncdocuments so this is pretty much a no-op.The concern in the code is making sure that the
GoCBDCPClientdid not change checkpoint names since we wish to resume a DCP feed across a version update.Pre-review checklist
fmt.Print,log.Print, ...)base.UD(docID),base.MD(dbName))docs/apiIntegration Tests
GSI=true,xattrs=truehttps://jenkins.sgwdev.com/job/SyncGatewayIntegration/279/