Skip to content

fix: Also migrate messages in PGP-contacts migration #6921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: link2xt/pgp-contacts
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 78 additions & 13 deletions src/sql/migrations.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
//! Migrations module.

use std::cmp::max;
use std::collections::BTreeMap;
use std::collections::BTreeSet;
use std::time::Instant;
Expand Down Expand Up @@ -1239,7 +1240,11 @@ CREATE INDEX gossip_timestamp_index ON gossip_timestamp (chat_id, fingerprint);
let start = Instant::now();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, this is the only place where we use Instant, maybe it's better to measure time with tools::Time as everywhere for uniformity

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instant is the easiest way to measure how long some operation takes, so, I think it's the best tool for the job here. We do use it in test_utils.rs, too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, Instant should be the right way to measure time in the ideal world. And we don't run tests on Android. But e.g. Context::background_fetch() uses tools::Time to measure and log elapsed time. It should be at least documented somewhere, which clock to use and when.

sql.execute_migration_transaction(|t| migrate_pgp_contacts(context, t), migration_version)
.await?;
info!(context, "PGP contacts migration took {:?}", start.elapsed());
info!(
context,
"PGP contacts migration took {:?} in total.",
start.elapsed()
);
}

let new_version = sql
Expand Down Expand Up @@ -1321,7 +1326,7 @@ fn migrate_pgp_contacts(
// Create up to 3 new contacts for every contact that has a peerstate:
// one from the Autocrypt key fingerprint, one from the verified key fingerprint,
// one from the secondary verified key fingerprint.
// In the process, build two maps from old contact id to new contact id:
// In the process, build maps from old contact id to new contact id:
// one that maps to Autocrypt PGP-contact, one that maps to verified PGP-contact.
let mut autocrypt_pgp_contacts: BTreeMap<u32, u32> = BTreeMap::new();
let mut autocrypt_pgp_contacts_with_reset_peerstate: BTreeMap<u32, u32> = BTreeMap::new();
Expand Down Expand Up @@ -1786,17 +1791,77 @@ fn migrate_pgp_contacts(
}

// ======================= Step 4: =======================
info!(
context,
"Marking contacts which remained in no chat at all as hidden: {orphaned_contacts:?}"
);
let mut mark_as_hidden_stmt = transaction
.prepare("UPDATE contacts SET origin=? WHERE id=?")
.context("Step 30")?;
for contact in orphaned_contacts {
mark_as_hidden_stmt
.execute((0x8, contact))
.context("Step 31")?;
{
info!(
context,
"Marking contacts which remained in no chat at all as hidden: {orphaned_contacts:?}"
);
let mut mark_as_hidden_stmt = transaction
.prepare("UPDATE contacts SET origin=? WHERE id=?")
.context("Step 30")?;
for contact in orphaned_contacts {
mark_as_hidden_stmt
.execute((0x8, contact))
.context("Step 31")?;
}
}

// ======================= Step 5: =======================
// Rewrite `from_id` in messages
{
let start = Instant::now();

let mut encrypted_msgs_stmt = transaction
.prepare(
"SELECT id, from_id, to_id
FROM msgs
WHERE id>9
AND (param LIKE '%\nc=1%' OR param LIKE 'c=1%')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not GLOB \"*c=*\"? GLOB is case-sensitive (though LIKE is also fine in this particular case)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know about GLOB, good to know! But since case-sensitivity doesn't matter here, I think it's fine to leave it as-is, since I tested it the way I wrote it, and would have to re-do this testing if I change it now.

FTR, SQL generally recommends to use ' rather than "

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine, but leaving this unresolved to maybe switch to GLOB if the PR will need retesting for some other reason

AND chat_id>9
ORDER BY id DESC LIMIT 10000",
)
.context("Step 32")?;
let mut rewrite_msg_stmt = transaction
.prepare("UPDATE msgs SET from_id=?, to_id=? WHERE id=?")
.context("Step 32.1")?;

struct LoadedMsg {
id: u32,
from_id: u32,
to_id: u32,
}

let encrypted_msgs = encrypted_msgs_stmt
.query_map((), |row| {
let id: u32 = row.get(0)?;
let from_id: u32 = row.get(1)?;
let to_id: u32 = row.get(2)?;
Ok(LoadedMsg { id, from_id, to_id })
})
.context("Step 33")?;

for msg in encrypted_msgs {
let msg = msg.context("Step 34")?;

let new_from_id = *autocrypt_pgp_contacts
.get(&msg.from_id)
.or_else(|| autocrypt_pgp_contacts_with_reset_peerstate.get(&msg.from_id))
.unwrap_or(&msg.from_id);

let new_to_id = *autocrypt_pgp_contacts
.get(&msg.to_id)
.or_else(|| autocrypt_pgp_contacts_with_reset_peerstate.get(&msg.to_id))
.unwrap_or(&msg.to_id);

rewrite_msg_stmt
.execute((new_from_id, new_to_id, msg.id))
.context("Step 35")?;
}
info!(
context,
"Rewriting msgs to PGP contacts took {:?}.",
start.elapsed()
);
}

Ok(())
Expand Down
Loading