Skip to content

mailserver_topics table should have a primary key using topic and chatId instead of only topic #2461

Open

Description

In #2460 I made a change so the last_request column is reset to 0 when deactivating a channel. This is done so it's possible to re-request the messages from a mailserver when you rejoin the channel again.

This is because this column is used to determine the from/to when requesting mailserver messages, and if that value is not updated, when you rejoin the channel, it will only attempt to retrieve data for the minutes that passed between leaving the channel and rejoining it again.

While this PR solved this issue, it surfaced a not so evident problem of having the topic being the primary key of the mailserver_topics table: Since topics are 4 bytes long, there exists a possibility of a collision by having more than one chatId belong to the same topic. It seems this was the original idea by creating the chat_ids column in the table, but it seems that this column is not used anywhere (it's always null), and also it would not fix the following issue:

Imagine you fixed the issue with the chat_ids column, and you join 'channel-1' and 'channel-2' which for this example, both end up with the topic '0xaabbccdd'. You end up with the following record in mailserver_topics:

topic      | chat_ids            | last_request | discovery | negotiated
0xaabbccdd | channel-1,channel-2 | 12334343     | 0         | 0  

If later you leave 'channel-1', last_request column will end up with 0, meaning that that 'channel-2' (which it is still being used), will need to sync from scratch again. I'm also not sure what would happen if I requests more messages for channel-2 before re-joining channel-1. I believe it's really easy to end up with an inconsistent history.

I propose that the following is done:

  1. Migrate mailserver_topics from this structure:
CREATE TABLE IF NOT EXISTS mailserver_topics (
    topic VARCHAR PRIMARY KEY,
    chat_ids VARCHAR,
    ...
) WITHOUT ROWID;

to

CREATE TABLE IF NOT EXISTS mailserver_topics (
    topic VARCHAR,
    chat_id VARCHAR,
    ...
    PRIMARY KEY(topic, chat_id)
) WITHOUT ROWID;
  1. When adding topics to that table, populate the chat_id column as well. This means that you'll end up with a record for each chat you have joined. (including channels not displayed like those used for status updates, contact codes, etc).
  2. When leaving a chat, you'd remove the record that matches the specific topic AND chat_id from the mailserver_topics table
  3. The logic for syncing topics would need to change a bit since it needs to take in account various possibly more than 1 record for a single topic, probably having to use the max/min last_request column when building the query for the mailserver.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions