feat: document store updates for multi-engine support - #8089
Open
nicpuppa wants to merge 7 commits into
Open
Conversation
…ocumentFactory (#6960) Adds an optional physicalTenantId field to DocumentCreationRequest and a shared Jackson reader-attribute key on DocumentFactory. Purely additive SDK surface needed by the multi-engine document-routing work that follows.
…#6960) Adds a Map<String, DocumentFactory>-based constructor overload to JacksonModuleDocumentDeserializer and its five sibling deserializers, alongside the existing single-DocumentFactory ones. Document.class deserialization resolves the correct physical tenant's factory from the DocumentFactory.PHYSICAL_TENANT_ID_ATTRIBUTE reader attribute, falling back to the sole configured tenant when the attribute is absent. This is additive only: existing single-factory call sites (including the proprietary-licensed connector-agentic-ai test module and the shared InboundConnectorContextBuilder/OutboundConnectorContextBuilder harness) are untouched.
…connectors (#6960) Adds PhysicalTenantIds.buildDocumentFactoriesByPhysicalTenantId, mirroring the existing per-physical-tenant SearchQueryClient builder: one DocumentFactory per configured physical tenant, each backed by its own CamundaDocumentStoreImpl/CamundaClient, with a single-client-only legacyDocumentFactory override escape hatch for existing tests/deployments. springInboundConnectorContextFactory now resolves each per-tenant delegate's DocumentFactory from this map instead of sharing one global instance across every physical tenant.
…enant attribute (#6960) InboundConnectorContextImpl.bindProperties/bindElementProperties now set DocumentFactory.PHYSICAL_TENANT_ID_ATTRIBUTE on the Jackson reader before deserializing, since the shared, Spring-singleton ObjectMapper backing those calls can't be rebuilt per physical tenant the way outbound's per-worker ObjectMapper already is. ConnectorsAutoConfiguration's connectorObjectMapper (used by inbound webhook/binding paths via @ConnectorsObjectMapper) now builds its JacksonModuleDocumentDeserializer from the per-physical-tenant DocumentFactory map instead of a single global instance.
Adds an end-to-end test proving that, with two camunda.clients.* configured, each physical tenant's DocumentFactory genuinely targets its own client: document creation attempts fail against each tenant's own distinct, unreachable REST address rather than silently succeeding through another tenant's client. Mirrors the lazy-CamundaClient pattern established by MultiClientPhysicalTenantWiringTest/MultiClientOutboundPhysicalTenantWiringTest so no live broker is needed.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds multi-engine document-store routing using physical tenant IDs across inbound contexts and Jackson deserialization.
Changes:
- Adds physical-tenant metadata and tenant-aware document-factory resolution.
- Routes inbound document operations to tenant-specific clients.
- Adds deserialization, resolution, and Spring wiring tests.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
connectors/aws/aws-bedrock/.../BedrockContentMapperTest.java |
Updates request construction. |
connector-sdk/core/.../DocumentFactory.java |
Defines the Jackson tenant attribute. |
connector-sdk/core/.../DocumentCreationRequest.java |
Adds physical tenant metadata. |
connector-runtime/spring-boot-starter-.../DocumentPhysicalTenantIsolationWiringTest.java |
Tests client isolation wiring. |
connector-runtime/spring-boot-starter-.../ConnectorsAutoConfiguration.java |
Configures tenant-aware deserialization. |
connector-runtime/jackson-datatype-document/.../JacksonModuleDocumentDeserializer.java |
Registers tenant-aware deserializers. |
connector-runtime/jackson-datatype-document/.../StringDeserializer.java |
Supports tenant-aware document strings. |
connector-runtime/jackson-datatype-document/.../ObjectDeserializer.java |
Supports tenant-aware object fields. |
connector-runtime/jackson-datatype-document/.../InputStreamDeserializer.java |
Supports tenant-aware streams. |
connector-runtime/jackson-datatype-document/.../DocumentDeserializer.java |
Resolves factories by tenant ID. |
connector-runtime/jackson-datatype-document/.../ByteArrayDeserializer.java |
Supports tenant-aware byte arrays. |
connector-runtime/connector-runtime-spring/.../PhysicalTenantIdResolutionTest.java |
Tests per-tenant factory construction. |
connector-runtime/connector-runtime-spring/.../PhysicalTenantIds.java |
Builds tenant-specific factories. |
connector-runtime/connector-runtime-spring/.../InboundConnectorRuntimeConfiguration.java |
Wires factories into inbound contexts. |
connector-runtime/connector-runtime-core/.../DocumentDeserializationPhysicalTenantIdTest.java |
Tests tenant-aware resolution behavior. |
connector-runtime/connector-runtime-core/.../InboundConnectorContextImpl.java |
Propagates tenant IDs during binding. |
nicpuppa
force-pushed
the
6961-outbound-multiengine-support
branch
from
July 27, 2026 13:37
c5fbf70 to
c2c545e
Compare
nicpuppa
force-pushed
the
6961-outbound-multiengine-support
branch
from
July 27, 2026 14:12
c2c545e to
f5df74a
Compare
nicpuppa
changed the title
nicpuppa
force-pushed
the
6961-outbound-multiengine-support
branch
from
July 27, 2026 14:15
f5df74a to
7e7e9fd
Compare
nicpuppa
marked this pull request as ready for review
July 27, 2026 14:16
nicpuppa
force-pushed
the
6961-outbound-multiengine-support
branch
2 times, most recently
from
July 27, 2026 16:22
597d135 to
14d50bf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Propagates the physical tenant through document creation and resolution, so multi-client deployments no longer share a single global document store across every configured client.
physicalTenantIdfield toDocumentCreationRequestand a sharedDocumentFactory.PHYSICAL_TENANT_ID_ATTRIBUTEJackson reader-attribute key, with a legacy constructor overload preserved for binary/source compatibility.DocumentFactory/CamundaDocumentStoreImplper configured physical tenant for inbound connectors (mirroring the existing per-tenantSearchQueryClient/ProcessInstanceClientpattern); outbound already had this via the sibling multi-engine work on this branch.Map<String, DocumentFactory>-based constructor overload to the Jackson document deserializers, resolving the correct tenant's factory from the reader attribute at deserialize time — needed because the shared, singletonObjectMapperbacking inbound property binding can't be rebuilt per tenant the way outbound's per-worker mapper can.InboundConnectorContextImpl.bindProperties/bindElementPropertiesandDefaultProcessInstanceContext.bind(intermediate/polling connectors).JobHandlerContext.create/InboundConnectorContextImpl.createnow stamp outgoingDocumentCreationRequests with their own physical tenant ID (never overriding an explicit connector-supplied value), andCamundaDocumentStoreImplvalidates it against its own configured tenant, catching a stale/cross-wiredDocumentFactoryinstead of silently writing to the wrong cluster.Related issues
closes #6960
Checklist
release, as this branch will be rebased onto main before the next release. Example backport labels:
backport stable/8.8: for changes that should be included in the next 8.8.x release.backport release-8.8.7: for changes that should be included in the specific release 8.8.7, and thisrelease has already been created. The release branch will be merged back into stable/8.8 later, so the change
will be included in future 8.8.x releases as well.