Skip to content
This repository was archived by the owner on Apr 26, 2024. It is now read-only.
This repository was archived by the owner on Apr 26, 2024. It is now read-only.

device_lists_changes_in_room is inefficient. #14037

Closed
@erikjohnston

Description

@erikjohnston

We see a lot of disk IO on the matrix.org DB go on reading from the device_lists_changes_in_room table.

I think this is due to the fact that we have a background process that processes each row, and then flips a boolean flag on the row (which is in an index). The causes the row to effectively get deleted/reinserted, meaning that a lot of the recent (ish) rows get shuffled around causing many more pages to have to be loaded from disk to service queries.

My proposal is to do one of:

  1. Have a separate table to keep track of devices that we need to process, and then delete out of that when we've processed a row. The disadvantage with this approach is that we double the amount of data we need to insert; or
  2. Remove the boolean flag and instead keep track of which stream ID we had processed up to. This has the disadvantage that we need to scan over every row of device_lists_changes_in_room table even though we may not need to handle them (i.e. we only need to process local device changes).

Metadata

Metadata

Assignees

Labels

A-Device-List-TrackingTelling clients about other devices. Often related to E2EE.A-PerformancePerformance, both client-facing and admin-facingO-FrequentAffects or can be seen by most users regularly or impacts most users' first experienceS-MinorBlocks non-critical functionality, workarounds exist.T-EnhancementNew features, changes in functionality, improvements in performance, or user-facing enhancements.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions