Skip to content

Comments

Sharded DCP: use common code#8074

Open
torcolvin wants to merge 5 commits intomainfrom
dcp-common-collections-code
Open

Sharded DCP: use common code#8074
torcolvin wants to merge 5 commits intomainfrom
dcp-common-collections-code

Conversation

@torcolvin
Copy link
Collaborator

Creates common code for sharded DCP to share between import and resync.

  • Create CollectionNames struct for common use. This type already exists in sgbucket.FeedArguments and probably could be pushed to sg-bucket in the future. It would probably be better to have a type that is map[string]map[string]{}struct in the future.
  • Add some error checking for ResyncManger options structs.
  • move ctx = CorrelationIDLogCtx(ctx, DCPImportFeedID) back to import listener in preparation for resync usage

Pre-review checklist

  • Removed debug logging (fmt.Print, log.Print, ...)
  • Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
  • Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Integration Tests

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors sharded DCP-related code to share common collection-name handling between import and resync, and tightens some resync manager option validation.

Changes:

  • Introduces base.CollectionNames (+ helpers) and migrates resync request/test usage to it.
  • Adds resync manager option type-checking to avoid panics on invalid options.
  • Moves import DCP correlation logging context setup back into the import listener and centralizes some DB context helpers (scope name, sharded DCP capability, collection name aggregation).

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
rest/indextest/index_test.go Updates resync payload construction to use base.NewCollectionNames.
rest/api.go Changes resync POST request body type for scopes to base.CollectionNames.
rest/adminapitest/collections_admin_api_test.go Updates admin API resync tests to use base.NewCollectionNames.
db/import_listener.go Refactors import feed startup to use shared helpers and sharded DCP decision logic.
db/database.go Adds scopeName, useShardedDCP(), and collectionNames() helpers; enforces single-scope DB config.
db/background_mgr_resync_dcp_test.go Updates resync DCP manager tests to pass base.CollectionNames.
db/background_mgr_resync_dcp.go Switches resync collection tracking to base.CollectionNames and adds option validation.
base/dcp_sharded.go Removes correlation ID context assignment from sharded DCP startup; updates comment wording.
base/collection_common.go Adds CollectionNames type and helper methods for common scope/collection mapping.

RIT3shSapata
RIT3shSapata previously approved these changes Feb 23, 2026
@torcolvin torcolvin self-assigned this Feb 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants