[Feature] Inbox pattern add support to binary and json

[lillo42]

Is your feature request related to a problem? Please describe.
The RDS inbox pattern uses a varchar to store the message. It would be nice if the Inbox pattern could offer to store messages as binary or JSON (or JSONB)

[ArmanPopat]

Hi,
I'd love to pick this up and have been looking into it. Just had a couple questions with regards to this that I'm hoping someone more familiar with the code base could answer.
I was thinking of storing the option for this in the RelationalDatabaseConfiguration class, however as not all the database providers have a native column type of JSON/JSONB, I'm not sure if this is too low level and whether there was another more appropriate place to store this (maybe database specific)?
My second question was where the implementation of this could live, I would want to extend the logic in the RelationalDatabaseInbox class, but as again JSON/JSONB types are not always supported, would this code need to live in the individual inbox implementations?

[iancooper]

Hi, I'd love to pick this up and have been looking into it. Just had a couple questions with regards to this that I'm hoping someone more familiar with the code base could answer. I was thinking of storing the option for this in the RelationalDatabaseConfiguration class, however as not all the database providers have a native column type of JSON/JSONB, I'm not sure if this is too low level and whether there was another more appropriate place to store this (maybe database specific)? My second question was where the implementation of this could live, I would want to extend the logic in the RelationalDatabaseInbox class, but as again JSON/JSONB types are not always supported, would this code need to live in the individual inbox implementations?

So, in the *OutboxBuilder code, we use an approach that modifies the DDL statement based on a flag. (see src/Paramore.Brighter.Outbox.MsSql/SqlOutboxBuilder.cs l.113 for an example). That builder is then available for a team to use to build their DB. ( See the example in samples/WebAPI/WebAPI_Common/DbMaker/SchemaCreation.cs, but our long-term thinking here is to move this to Aspire rather than have you build it.)

Then in IAmARelationalDatabaseConfiguration, we set a value to indicate that we are using a binary payload for the Outbox BinaryMessagePayload. Then in the RelationalDatabaseOutbox (src/Paramore.Brighter/RelationDatabaseOutbox.cs) within InitAddDbParameters(l.1288) we configure the parameters differently depending on whether your payload is binary.

Now the Inbox code works similarly. So, in the *InboxBuilder code, we use an approach that modifies the DDL statement based on a flag. (see src/Paramore.Brighter.Inbox.MySql/MySqlInboxBuilder.cs l.473 for an example). That builder is then available for a team to use to build their DB. ( See the example in src/Paramore.Brighter.Inbox.MySql/MySqlInboxBuilder.cs, l.251 but our long-term thinking here is to move this to Aspire rather than have you build it.)

Then in IAmARelationalDatabaseConfiguration, we also use the value BinaryMessagePayload to indicate that we are using a binary payload for the Outbox. Then in the RelationalDatabaseInbox (src/Paramore.Brighter/RelationalDatabaseInbox.cs) within CreateAddParameters (l.428) we use that value to support a binary payload.

I would modify @lillo42's statement to say that we support binary payloads in both Outbox and Inbox for relational stores.

So, in this case, the modification would be to use JSON or JSONB types instead of varchar and binary where supported.

To do that, I think we would need a flag, such as JsonMessagePayload. Then if you set that flag, we could, where supported, use the JSON and JSONB types instead of VarChar and Binary. We would need additional DDL statements. We should throw a Configuration exception if not available on that Db type.

For clarity, we MUST NOT remove the existing BinaryMessagePayload and associated code. This would be a breaking change, so even if we thought it was a good idea, we could not do it until V11 now. So this would be a new flag for folks who want that option.

I presume that the value proposition is to make it easier to query the outbox and inbox, where payloads are JSON. If that helps, we could add this. But we must not replace the existing approach.

[iancooper]

PS @ArmanPopat Sorry for the slow reply on this one. I have been a little busy over the last couple of weeks.

[ArmanPopat]

Hi @iancooper, sorry for getting back to this after quite a while. I've created a pr based on the recommendation above (#3956).

Not sure if I was required to create an ADR for this?

[iancooper]

@ArmanPopat Generally, an ADR is good, so we know what the code that you are submitting is supposed to do and how. It makes it much easier to review. You could create one from my comment above.