Encrypted appservices: how to?

[matrixbot]

This issue has been migrated from #10653.

---

The end goal of this adventure is to have an appservice which can participate in encrypted rooms. While this doesn't necessarily provide end-to-end encryption, it does mean that encrypted messages can be decrypted by application services. The driving usecase is largely private context (DM/ping me internally), but is effectively a high-traffic single-user highly reliable bot using the appservice API (sync won't keep up). Other usecases like bridges are worth considering, even if not driving this work, due to the fact that someone will try it despite warnings.

There's 3-4 major pieces in order to get this support working:

• Sending to-device messages to appservices for their interested users. This has security context to consider.
• Sending device list changes to appservices.
• Sending One Time Key (OTK) counts to appservices for each of their interested users.
• Sending fallback key usage to appservices for each of their interested users (we could probably go without this, but would rather not).

Other considerations, like setting up key backups, are already largely solved due to the masquerading support on the endpoints. Account data changes might need to make it down to appservices, but this can be considered future work for the purposes of this conversation.

Sending device messages (to-device)

This is effectively sending ephemeral events to appservices, which Synapse already supports for read receipts, typing, and presence. Currently the streams appear to operate off a from and to token system, grabbing a batch of events between those tokens with some filter criteria, however for appservices the filter criteria is a bit more nuanced. Specifically, the appservice can easily get to millions of users under its namespace which means millions of inboxes to check. We could add an array of user IDs to the DB fetch function for inboxes to check, however this can, again, be millions of entries long: this would be bad for the DB server.

Appservices already have to care about implicit versus explicit user reservations and will likely have to do additional filtering based upon that, so the proposal I have is largely that the appservice handler grab all device messages for all inboxes between the stream tokens, then filter that down to interested users in code. The function would build two piles: implicit interest and explicit interest. Explicit interest would cause the function to delete those messages from the inbox as they'll shortly be "delivered". Implicit interest means those messages won't be deleted, but will still be sent to the appservice. This is where the security context comes into play: this provides a way for appservices to theoretically intercept device messages without the user knowing. However, the server admin will have had to approve the appservice implicitly by installing it, so it might be fine. This might need more discussion.

An issue with deleting the device messages is reliability: where the code appears to build the transaction and where that transaction is sent to the appservice is disjointed. This isn't important for things like presence, typing, or read receipts (to a degree), however for device messages if the server were to be restarted while an undelivered transaction is in the queue then the messages will be lost, leading to UISIs (in this use case).

If Synapse's streams send events and not just stream positions over replication then the appservice handler might be able to just queue the device messages into the transaction straight from there, thus not having to bother with any stream ID nonsense. However, this assumes that replication is reliable and that the events are in fact sent over replication.

Sending device list changes

The current proposal for this is to send device list changes through at the top level of the transaction: matrix-org/matrix-spec-proposals#3202

Filtering changes is relatively easy as the logic should already exist for presence. Determining when a device list has left the appservice's point of view is a bit more of a challenge, I think, as it'd mean tying the appservice handling to membership events and running a fairly expensive loop of checking all the rooms the user was in prior to the leave and matching those rooms against appservice interest.

For the purposes of determining interest, usually this sort of thing would be checked against whether the appservice was interested in the room or not, however given the intended use case of device lists it seems fair to ensure the appservice has a relevant member in the room which would trigger interest instead. This changes the check to determine if the appservice no longer shares a room with the user after the leave event, which might be even more expensive but involve less spam?

This also has the potential to be the same problem as device messages: reliability during downtime of the appservice is diminished.

OTKs & fallback keys

This is an unsolved problem as of writing. The MSC has a thread to say that this information shouldn't be sent over in a similar fashion to /sync (which always presents it for the user) due to performance concerns: if it always included the information, there'd be millions of objects in the JSON to worry about. Instead, the suggestion is to include the field when the counts/fallback key usage changes. I have no idea if this is even possible to detect in Synapse, but the cheap solution would be to include the fields if the relevant user is receiving a device message.

Possible alternatives

We could invest even more heavily in /sync, however that doesn't help the encrypted bridges use case. A million sync streams sounds worse than an expensive appservice loop, but neither is going to perform well anyways.