Repository navigation
AWS SQS: add broker-per-tenant multi-tenancy (dedicated account/region per tenant) #3304
Description
Scope
SQS already supports named brokers (AddNamedAmazonSqsBroker, AmazonSqsTransportExtensions.cs:55). This issue adds the missing broker-per-tenant feature: each Wolverine tenant gets its own dedicated SQS client (distinct AWS credentials/account, region, or ServiceURL) while sharing queue topology, routed at runtime by Envelope.TenantId.
SQS connection model
AmazonSqsTransport : BrokerTransport<AmazonSqsQueue> already has the (string protocol) ctor (:27) and inherits TenantedIdBehavior. Connection config is one field AmazonSQSConfig Config (:99, holds ServiceURL/RegionEndpoint) plus Func<IWolverineRuntime, AWSCredentials>? CredentialSource (:93). Client built lazily in BuildClient (:327-336), stored in internal IAmazonSQS? Client (:101), assigned in ConnectAsync (:213). The main refactor point: senders/listeners hard-reference _parent.Client — there is no per-client parametrization today.
Follow the Azure Service Bus model (AzureServiceBusTenant holds a child transport whose Compile(parent) copies credentials) — the closest analog since AWS is credential/region based.
Execution plan
New file — Wolverine.AmazonSqs/Internal/AmazonSqsTenant.cs: TenantId + a child AmazonSqsTransport (own Config, CredentialSource, Client, and its own Queues cache) with Compile(AmazonSqsTransport parent) copying CredentialSource/region defaults (mirror AzureServiceBusTenant.Compile). The child-transport-per-tenant model is cleaner than sharing one AmazonSqsQueue, because QueueUrl is cached per endpoint and would collide across tenants.
Changed files:
Internal/AmazonSqsTransport.cs— addLightweightCache<string, AmazonSqsTenant> Tenants; inConnectAsync(:213) also build each tenant'sClientand provision tenant queues; dispose tenant clients.Internal/AmazonSqsQueue.cs— refactorCreateSender(:397) andBuildListenerAsync(:380) to route through a specificIAmazonSQSclient (or tenant transport) rather than hard-referencing_parent.Client. Then: ifTenants.Any() && TenancyBehavior == TenantAware, build aTenantedSender(per-tenant sender bound to the tenant's client) and aCompoundListener(default + per-tenantSqsListenerwrapped inReceiverWithRules+TenantIdRule). Simplest structure: move sender/listener building onto the transport as partial classes (as Azure did) so each tenant's child transport builds with its own client.Internal/SqsListener.cs,SqsSenderProtocol.cs,InlineSqsSender.cs— parametrize theIAmazonSQSclient (today they pull_transport.Client).AmazonSqsTransportConfiguration.cs— new public API (below).
New public API (AmazonSqsTransportConfiguration)
AmazonSqsTransportConfiguration TenantIdBehavior(TenantedIdBehavior behavior);
AmazonSqsTransportConfiguration AddTenant(string tenantId, Action<AmazonSQSConfig> configure); // dedicated region/endpoint
AmazonSqsTransportConfiguration AddTenant(string tenantId, AWSCredentials credentials, Action<AmazonSQSConfig>? configure = null); // dedicated accountAWS-specific caveats
- Per-tenant axes: credentials (account), region, ServiceURL (endpoint). Queue provisioning (
InitializeAsync/SetupAsync,QueueUrlresolution) must run against the tenant's client — hence the child-transport-per-tenant model (each tenant gets its ownQueuescache and queue-URL cache). - DLQs and system/response queues (
tryBuildSystemEndpoints,:179) are per-client — tenants in separate accounts need their own DLQ/response queues. - FIFO/fair-queue group-id logic is client-agnostic — no change.
Test coverage
LocalStack is a single container (port 4566); it does not enforce account boundaries, so tenant tests cannot prove separate accounts. Simulate tenant isolation via a different region on the same LocalStack endpoint (LocalStack accepts any region) or distinct queue-name prefixes; assert message routing + Envelope.TenantId stamping rather than physical broker separation (contrast NATS which spins a real second broker). Add unit tests (no LocalStack) asserting each tenant's child transport gets the right region/endpoint/credentials and that CreateSender returns a TenantedSender with the expected registered senders (extend AmazonSqsTransportTests.cs).
Scenarios: tenant publish → tenant queue/region; default → shared; tenant listener stamps TenantId; TenantedIdBehavior variants; per-tenant queue + DLQ provisioning.
Docs
docs/guide/messaging/transports/sqs/index.md already has a "Connecting to Multiple Brokers" section (:112). Add a broker-per-tenant section here (or a new sqs/multi-tenancy.md) with a snippet sourced from a new Samples/Bootstrapping.cs region (mirror the named-broker snippet reference).
Background — the two multi-broker features
PR #3283 hardened the NATS transport and, in the process, brought NATS up to parity with RabbitMQ/Azure Service Bus on broker-per-tenant multi-tenancy. This issue is part of a sweep to bring the remaining transports up to parity on the two established multi-broker features:
- Named broker —
AddNamed<X>Broker(BrokerName name, …)registers an additional, independent transport instance of the same broker type alongside the default, so endpoints can be pinned to a specific broker by name (…OnNamedBrokeroverloads). Reference implementations: RabbitMQ, Kafka, AWS SQS, Azure Service Bus. - Broker-per-tenant —
AddTenant(tenantId, …)gives each tenant its own dedicated connection/broker while sharing endpoint topology; outbound uses the frameworkTenantedSender, inbound uses aCompoundListener(one listener per tenant connection, stamping the tenant id viaTenantIdRule). Reference implementations: RabbitMQ, Azure Service Bus, and now NATS (NATS transport: dynamic subjects, JetStream dedup, per-tenant connection #3283).
Current capability matrix
| Transport | Named broker | Broker-per-tenant |
|---|---|---|
| RabbitMQ | ✅ | ✅ |
| Azure Service Bus | ✅ | ✅ |
| NATS | ❌ | ✅ (#3283) |
| Kafka | ✅ | ❌ |
| AWS SQS | ✅ | ❌ |
| AWS SNS | ❌ | ❌ |
| GCP Pub/Sub | ❌ | ❌ |
| MQTT | ❌ | ❌ |
| Pulsar | ❌ | ❌ |
| Redis | ❌ | ❌ |
| SignalR | N/A | N/A |
Framework building blocks (already exist — reuse, do not rebuild)
Named broker:
BrokerName—record BrokerName(string Name)(src/Wolverine/BrokerName.cs).TransportCollection.GetOrCreate<T>(BrokerName? name)(src/Wolverine/TransportCollection.cs:74-99). Whenname != nullit doesActivator.CreateInstance(typeof(T), name.Name)and registers the transport keyed by itsProtocol. Two consequences: the transport must expose a publicT(string protocol)constructor, and the named broker'sProtocolbecomes its URI scheme (othername://…), so endpoint URIs must be built fromparent.Protocol, not a hard-coded literal.
Broker-per-tenant:
TenantedSender(src/Wolverine/Transports/Sending/TenantedSender.cs) — routesSendAsynconenvelope.TenantId;TenantedIdBehaviorenum (TenantIdRequired/IgnoreUnknownTenants/FallbackToDefault). Must NOT implementISenderRequiresCallback(outbox-deletion regression Message is stuck in outbox with multiple tenants and durable messaging #2361).CompoundListener(src/Wolverine/Transports/IListener.cs:17) — aggregates inner listeners; per-envelope completion routes back over the receiving connection viaEnvelope.Listener(it throws on its ownCompleteAsync).TenantIdRule(src/Wolverine/IEnvelopeRule.cs:42) +ReceiverWithRules(src/Wolverine/Transports/IReceiver.cs:16) — stamp inboundEnvelope.TenantId.Endpoint.TenancyBehavior(src/Wolverine/Configuration/Endpoint.cs:230, defaultTenantAware) +BrokerTransport.TenantedIdBehavior(src/Wolverine/Transports/BrokerTransport.cs:28). Every fan-out site guards onTenants.Any() && endpoint.TenancyBehavior == TenantAware.
Canonical reference files to copy from
- Named broker:
RabbitMqTransportExtensions.cs:21-26,75-83,187-199;AmazonSqsTransportExtensions.cs:17-22,55-74; testWolverine.RabbitMQ.Tests/end_to_end_with_named_broker.cs. - Broker-per-tenant:
RabbitMqTransportExpression.cs:45-78+RabbitMqTenant.cs+RabbitMqTransport.cs:68,236-239,429-474; NATSNatsTransportExpression.cs:195-332+NatsTenant.cs+NatsTransport.cs:33,160-244+NatsEndpoint.cs:142-283; testsWolverine.Nats.Tests/NatsPerTenantConnectionTests.cs+NatsTestHelpers.cs.
Common acceptance criteria (all issues)
- New public API mirrors the naming conventions of the reference transports.
- Full green
dotnet build wolverine.slnx -c Release. - New integration tests proving: (a) a
DeliveryOptions { TenantId = … }/…OnNamedBrokermessage lands on the correct broker and NOT the shared/default one, (b) a default message lands on the shared/default broker, (c) for tenant work, an inbound tenant message is consumed and stamped with itsTenantId. - Docs section added under
docs/guide/messaging/transports/…with a compilingsnippetsample.
Activity
Merge pull request #3316 from JasperFx/feat/sqs-broker-per-tenant-3304