Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide CryptoStore::mark_inbound_group_sessions_as_backed_up on stores and use it in BackupMachine::mark_request_as_sent #2934

Merged
merged 10 commits into from
Dec 15, 2023

Conversation

andybalaam
Copy link
Contributor

@andybalaam andybalaam commented Dec 12, 2023

Fixes element-hq/element-web#26488

I am not certain I've added a public API, but am guessing I have, so I documented it in the changelog.

  • Public API changes documented in changelogs (optional)

Previously, in BackupMachine::mark_request_as_sent we fetched, deserialised and decrypted all existing sessions by calling get_inbound_group_sessions. Now, via a new method CryptoStore::mark_inbound_group_sessions_as_backed_up, we avoid fetching any sessions except the ones we actually want to mark as backed up, and we mark them as backed up without decrypting them at all.

@andybalaam andybalaam force-pushed the andybalaam/mark_sessions_as_backed_up branch 2 times, most recently from 81dad3c to f58f34f Compare December 12, 2023 16:05
@andybalaam andybalaam marked this pull request as ready for review December 12, 2023 16:08
@andybalaam andybalaam requested a review from a team as a code owner December 12, 2023 16:08
@andybalaam andybalaam requested review from bnjbvr and removed request for a team December 12, 2023 16:08
…in BackupMachine::mark_request_as_sent

Signed-off-by: Andy Balaam <andy.balaam@matrix.org>
@andybalaam andybalaam force-pushed the andybalaam/mark_sessions_as_backed_up branch from f58f34f to 4ebb8e2 Compare December 12, 2023 16:09
@andybalaam andybalaam changed the title Provide CryptoStore::mark_sessions_as_backed_up on stores and use it in BackupMachine::mark_request_as_sent Provide CryptoStore::mark_inbound_group_sessions_as_backed_up on stores and use it in BackupMachine::mark_request_as_sent Dec 12, 2023
Copy link

codecov bot commented Dec 12, 2023

Codecov Report

Attention: 11 lines in your changes are missing coverage. Please review.

Comparison is base (614bf94) 82.86% compared to head (f3101ba) 82.84%.

Files Patch % Lines
crates/matrix-sdk-sqlite/src/utils.rs 55.55% 8 Missing ⚠️
crates/matrix-sdk-sqlite/src/crypto_store.rs 89.47% 2 Missing ⚠️
crates/matrix-sdk-crypto/src/store/traits.rs 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2934      +/-   ##
==========================================
- Coverage   82.86%   82.84%   -0.02%     
==========================================
  Files         220      220              
  Lines       22554    22583      +29     
==========================================
+ Hits        18689    18710      +21     
- Misses       3865     3873       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@BillCarsonFr BillCarsonFr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Hywan Hywan changed the title Provide CryptoStore::mark_inbound_group_sessions_as_backed_up on stores and use it in BackupMachine::mark_request_as_sent Provide CryptoStore::mark_inbound_group_sessions_as_backed_up on stores and use it in BackupMachine::mark_request_as_sent Dec 13, 2023
Copy link
Member

@Hywan Hywan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is code duplication with an existing code (chunk_large_query_over and repeat_vars) in matrix-sdk-sqlite. Reading the existing code, I've found a bug (see the comments). Ideally, you can refactor your PR to use the existing code, fix the bugs, re-use your tests if you feel that's necessary (I believe the previous code wasn't unit tested).

Apart from that, the rest of the PR looks good to me.

crates/matrix-sdk-crypto/src/backups/mod.rs Outdated Show resolved Hide resolved
crates/matrix-sdk-sqlite/src/crypto_store.rs Outdated Show resolved Hide resolved
crates/matrix-sdk-sqlite/src/crypto_store.rs Outdated Show resolved Hide resolved
crates/matrix-sdk-sqlite/src/utils.rs Outdated Show resolved Hide resolved
@bnjbvr bnjbvr removed their request for review December 14, 2023 10:12
@andybalaam
Copy link
Contributor Author

There is code duplication with an existing code (chunk_large_query_over and repeat_vars) in matrix-sdk-sqlite. Reading the existing code, I've found a bug (see the comments). Ideally, you can refactor your PR to use the existing code, fix the bugs, re-use your tests if you feel that's necessary (I believe the previous code wasn't unit tested).

Apart from that, the rest of the PR looks good to me.

Great, thanks - I will refactor to use chunk_large_query_over


let session_ids_len = session_ids.len();

self.chunk_large_query_over(session_ids, None, move |session_ids| {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hywan I've attempted to refactor it to use chunk_large_query_over - what do you think?

I couldn't find a way to do it in one transaction - I'm not sure whether that's needed or not.

I think I would need to add a new with_transaction_sync and that would need a new interact_async - do you agree?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with @richvdh and @BillCarsonFr we think not having it all in a single transaction is OK.

Copy link
Member

@Hywan Hywan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👌

@andybalaam andybalaam merged commit b21a438 into main Dec 15, 2023
35 checks passed
@andybalaam andybalaam deleted the andybalaam/mark_sessions_as_backed_up branch December 15, 2023 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Element R: App freezes while backing up megolm keys
3 participants