Skip to content

AWS SQS: add broker-per-tenant multi-tenancy (dedicated account/region per tenant) #3304

Closed

Description

@jeremydmiller

Scope

SQS already supports named brokers (AddNamedAmazonSqsBroker, AmazonSqsTransportExtensions.cs:55). This issue adds the missing broker-per-tenant feature: each Wolverine tenant gets its own dedicated SQS client (distinct AWS credentials/account, region, or ServiceURL) while sharing queue topology, routed at runtime by Envelope.TenantId.

SQS connection model

AmazonSqsTransport : BrokerTransport<AmazonSqsQueue> already has the (string protocol) ctor (:27) and inherits TenantedIdBehavior. Connection config is one field AmazonSQSConfig Config (:99, holds ServiceURL/RegionEndpoint) plus Func<IWolverineRuntime, AWSCredentials>? CredentialSource (:93). Client built lazily in BuildClient (:327-336), stored in internal IAmazonSQS? Client (:101), assigned in ConnectAsync (:213). The main refactor point: senders/listeners hard-reference _parent.Client — there is no per-client parametrization today.

Follow the Azure Service Bus model (AzureServiceBusTenant holds a child transport whose Compile(parent) copies credentials) — the closest analog since AWS is credential/region based.

Execution plan

New file — Wolverine.AmazonSqs/Internal/AmazonSqsTenant.cs: TenantId + a child AmazonSqsTransport (own Config, CredentialSource, Client, and its own Queues cache) with Compile(AmazonSqsTransport parent) copying CredentialSource/region defaults (mirror AzureServiceBusTenant.Compile). The child-transport-per-tenant model is cleaner than sharing one AmazonSqsQueue, because QueueUrl is cached per endpoint and would collide across tenants.

Changed files:

  • Internal/AmazonSqsTransport.cs — add LightweightCache<string, AmazonSqsTenant> Tenants; in ConnectAsync (:213) also build each tenant's Client and provision tenant queues; dispose tenant clients.
  • Internal/AmazonSqsQueue.cs — refactor CreateSender (:397) and BuildListenerAsync (:380) to route through a specific IAmazonSQS client (or tenant transport) rather than hard-referencing _parent.Client. Then: if Tenants.Any() && TenancyBehavior == TenantAware, build a TenantedSender (per-tenant sender bound to the tenant's client) and a CompoundListener (default + per-tenant SqsListener wrapped in ReceiverWithRules+TenantIdRule). Simplest structure: move sender/listener building onto the transport as partial classes (as Azure did) so each tenant's child transport builds with its own client.
  • Internal/SqsListener.cs, SqsSenderProtocol.cs, InlineSqsSender.cs — parametrize the IAmazonSQS client (today they pull _transport.Client).
  • AmazonSqsTransportConfiguration.cs — new public API (below).

New public API (AmazonSqsTransportConfiguration)

AmazonSqsTransportConfiguration TenantIdBehavior(TenantedIdBehavior behavior);
AmazonSqsTransportConfiguration AddTenant(string tenantId, Action<AmazonSQSConfig> configure); // dedicated region/endpoint
AmazonSqsTransportConfiguration AddTenant(string tenantId, AWSCredentials credentials, Action<AmazonSQSConfig>? configure = null); // dedicated account

AWS-specific caveats

  • Per-tenant axes: credentials (account), region, ServiceURL (endpoint). Queue provisioning (InitializeAsync/SetupAsync, QueueUrl resolution) must run against the tenant's client — hence the child-transport-per-tenant model (each tenant gets its own Queues cache and queue-URL cache).
  • DLQs and system/response queues (tryBuildSystemEndpoints, :179) are per-client — tenants in separate accounts need their own DLQ/response queues.
  • FIFO/fair-queue group-id logic is client-agnostic — no change.

Test coverage

LocalStack is a single container (port 4566); it does not enforce account boundaries, so tenant tests cannot prove separate accounts. Simulate tenant isolation via a different region on the same LocalStack endpoint (LocalStack accepts any region) or distinct queue-name prefixes; assert message routing + Envelope.TenantId stamping rather than physical broker separation (contrast NATS which spins a real second broker). Add unit tests (no LocalStack) asserting each tenant's child transport gets the right region/endpoint/credentials and that CreateSender returns a TenantedSender with the expected registered senders (extend AmazonSqsTransportTests.cs).

Scenarios: tenant publish → tenant queue/region; default → shared; tenant listener stamps TenantId; TenantedIdBehavior variants; per-tenant queue + DLQ provisioning.

Docs

docs/guide/messaging/transports/sqs/index.md already has a "Connecting to Multiple Brokers" section (:112). Add a broker-per-tenant section here (or a new sqs/multi-tenancy.md) with a snippet sourced from a new Samples/Bootstrapping.cs region (mirror the named-broker snippet reference).


Background — the two multi-broker features

PR #3283 hardened the NATS transport and, in the process, brought NATS up to parity with RabbitMQ/Azure Service Bus on broker-per-tenant multi-tenancy. This issue is part of a sweep to bring the remaining transports up to parity on the two established multi-broker features:

  1. Named broker — AddNamed<X>Broker(BrokerName name, …) registers an additional, independent transport instance of the same broker type alongside the default, so endpoints can be pinned to a specific broker by name (…OnNamedBroker overloads). Reference implementations: RabbitMQ, Kafka, AWS SQS, Azure Service Bus.
  2. Broker-per-tenant — AddTenant(tenantId, …) gives each tenant its own dedicated connection/broker while sharing endpoint topology; outbound uses the framework TenantedSender, inbound uses a CompoundListener (one listener per tenant connection, stamping the tenant id via TenantIdRule). Reference implementations: RabbitMQ, Azure Service Bus, and now NATS (NATS transport: dynamic subjects, JetStream dedup, per-tenant connection #3283).

Current capability matrix

Transport Named broker Broker-per-tenant
RabbitMQ ✅ ✅
Azure Service Bus ✅ ✅
NATS ❌ ✅ (#3283)
Kafka ✅ ❌
AWS SQS ✅ ❌
AWS SNS ❌ ❌
GCP Pub/Sub ❌ ❌
MQTT ❌ ❌
Pulsar ❌ ❌
Redis ❌ ❌
SignalR N/A N/A

Framework building blocks (already exist — reuse, do not rebuild)

Named broker:

  • BrokerName — record BrokerName(string Name) (src/Wolverine/BrokerName.cs).
  • TransportCollection.GetOrCreate<T>(BrokerName? name) (src/Wolverine/TransportCollection.cs:74-99). When name != null it does Activator.CreateInstance(typeof(T), name.Name) and registers the transport keyed by its Protocol. Two consequences: the transport must expose a public T(string protocol) constructor, and the named broker's Protocol becomes its URI scheme (othername://…), so endpoint URIs must be built from parent.Protocol, not a hard-coded literal.

Broker-per-tenant:

  • TenantedSender (src/Wolverine/Transports/Sending/TenantedSender.cs) — routes SendAsync on envelope.TenantId; TenantedIdBehavior enum (TenantIdRequired / IgnoreUnknownTenants / FallbackToDefault). Must NOT implement ISenderRequiresCallback (outbox-deletion regression Message is stuck in outbox with multiple tenants and durable messaging #2361).
  • CompoundListener (src/Wolverine/Transports/IListener.cs:17) — aggregates inner listeners; per-envelope completion routes back over the receiving connection via Envelope.Listener (it throws on its own CompleteAsync).
  • TenantIdRule (src/Wolverine/IEnvelopeRule.cs:42) + ReceiverWithRules (src/Wolverine/Transports/IReceiver.cs:16) — stamp inbound Envelope.TenantId.
  • Endpoint.TenancyBehavior (src/Wolverine/Configuration/Endpoint.cs:230, default TenantAware) + BrokerTransport.TenantedIdBehavior (src/Wolverine/Transports/BrokerTransport.cs:28). Every fan-out site guards on Tenants.Any() && endpoint.TenancyBehavior == TenantAware.

Canonical reference files to copy from

  • Named broker: RabbitMqTransportExtensions.cs:21-26,75-83,187-199; AmazonSqsTransportExtensions.cs:17-22,55-74; test Wolverine.RabbitMQ.Tests/end_to_end_with_named_broker.cs.
  • Broker-per-tenant: RabbitMqTransportExpression.cs:45-78 + RabbitMqTenant.cs + RabbitMqTransport.cs:68,236-239,429-474; NATS NatsTransportExpression.cs:195-332 + NatsTenant.cs + NatsTransport.cs:33,160-244 + NatsEndpoint.cs:142-283; tests Wolverine.Nats.Tests/NatsPerTenantConnectionTests.cs + NatsTestHelpers.cs.

Common acceptance criteria (all issues)

  • New public API mirrors the naming conventions of the reference transports.
  • Full green dotnet build wolverine.slnx -c Release.
  • New integration tests proving: (a) a DeliveryOptions { TenantId = … } / …OnNamedBroker message lands on the correct broker and NOT the shared/default one, (b) a default message lands on the shared/default broker, (c) for tenant work, an inbound tenant message is consumed and stamped with its TenantId.
  • Docs section added under docs/guide/messaging/transports/… with a compiling snippet sample.

Activity

  1. added a commit that references this issue on Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    awsdocumentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions