Skip to content

Prune cardano transactions in signer after block range roots computation #1685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
May 17, 2024

Conversation

Alenar
Copy link
Collaborator

@Alenar Alenar commented May 16, 2024

Content

This PR add, to the signer only, a pruning of the cardano transactions after the computation of block ranges roots.

This is done using a decorator wrapping the existing CardanoTransactionsImporter.

Two new configuration parameters have been added to signer Configuration structure:

  • network_security_parameter, default to 2160 (mainnet value).
  • enable_transaction_pruning, default to true.

Since both have a default value our signer won't have to update their configurations.

Pre-submit checklist

  • Branch
    • Tests are provided (if possible)
    • Crates versions are updated (if relevant)
    • CHANGELOG file is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • No clippy warnings in the CI
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested
  • Documentation
    • Update README file (if relevant)
    • Update documentation website (if relevant)
    • Add dev blog post (if relevant)

Comments

  • As it was the third time that we needed to do a simple query to retrieve a single result from database, a system using a extension trait on the sqlite connection has been added to do that simply with reduced boilerplate.
  • Some simplification has been done on the configuration system: a leaner way to hydrate default configuration (mithril-signer & mithril-aggregator) and generalized use of the test configuration values (new_sample(party_id), mithril-signer).

Issue(s)

Closes #1645

@Alenar Alenar requested review from sfauvel, jpraynaud and dlachaume May 16, 2024 14:05
Copy link

github-actions bot commented May 16, 2024

Test Results

    3 files  ± 0     43 suites  ±0   8m 32s ⏱️ ±0s
  970 tests +11    970 ✅ +11  0 💤 ±0  0 ❌ ±0 
1 064 runs  +11  1 064 ✅ +11  0 💤 ±0  0 ❌ ±0 

Results for commit e258352. ± Comparison against base commit 2fe74c2.

♻️ This comment has been updated with latest results.

@Alenar Alenar force-pushed the djo/1645/prune_ctx_in_signer branch 2 times, most recently from 72360bf to 14be502 Compare May 17, 2024 10:44
@Alenar Alenar temporarily deployed to testing-preview May 17, 2024 10:52 — with GitHub Actions Inactive
@Alenar Alenar temporarily deployed to testing-sanchonet May 17, 2024 10:52 — with GitHub Actions Inactive
Copy link
Collaborator

@dlachaume dlachaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

/// Extension trait for the [SqliteConnection] type.
pub trait ConnectionExtensions {
/// Execute the given sql query and return the value of the first cell read.
fn query_single_cell<Q: Into<String>, T: ReadableWithIndex>(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the use of the generic data type Q: Into<String> necessary here? The function is always called with &str parameters. Would it be simpler to directly use &str instead?

Copy link
Collaborator Author

@Alenar Alenar May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's a low level api that may be used in different kind of scenario I would like to have some flexibility with the input parameters, hence the generic.
That said after re-challenging it I don't need a owned string here but only a ref to a str, so I'm thinking of changing the generics to AsRef<str>, this avoid an allocation and simplify a bit the implementation:

 impl ConnectionExtensions for SqliteConnection {
-    fn query_single_cell<Q: Into<String>, T: ReadableWithIndex>(
+    fn query_single_cell<Q: AsRef<str>, T: ReadableWithIndex>(
         &self,
         sql: Q,
         params: &[Value],
     ) -> StdResult<T> {
-        let sql = sql.into();
         let mut statement = self.prepare(&sql).with_context(|| {
             format!(
                 "Prepare query error: SQL=`{}`",
-                &sql.replace('\n', " ").trim()
+                sql.as_ref().replace('\n', " ").trim()
             )
         })?;
         statement.bind(params)?;

@Alenar Alenar temporarily deployed to testing-preview May 17, 2024 13:06 — with GitHub Actions Inactive
@Alenar Alenar temporarily deployed to testing-sanchonet May 17, 2024 13:06 — with GitHub Actions Inactive
Alenar added 10 commits May 17, 2024 16:13
By using a local function to avoid repeating the namespace and simplify
the conversion into a `config::Value`.
By using `new_sample()` everywhere it make sense instead of building the
whole struct each time.
For now it define one method, `query_single_cell`, that allow to read a
single value from the database.
The two cases in the ctx repository where a simple query were needed
instead of the SQliteEntity system has been simplified using this
method.
A first request obtains the highest `block_range_root.start` and a
second use it, and a given number of block to keep, as the prune
threshold.

We do that this instead of using a join to make it simple to test (else
the tests would need data on the two tables, our test framework doesn't
allow yet to do that without a lot of boilerplate).
…igs to signer

Also set a default value to them of `2160` (mainnet value) and `true`
(enable pruning by default).
It would happen if the number of block to keep is greater than the
highest stored block range root start.
By using a `AsRef<str>` instead of a `Into<String>`. This remove the
need to locally allocate the string while keeping flexibility for the
caller.
@Alenar Alenar force-pushed the djo/1645/prune_ctx_in_signer branch from 898f444 to 4164aca Compare May 17, 2024 14:13
@Alenar Alenar temporarily deployed to testing-preview May 17, 2024 14:20 — with GitHub Actions Inactive
@Alenar Alenar temporarily deployed to testing-sanchonet May 17, 2024 14:20 — with GitHub Actions Inactive
Copy link
Member

@jpraynaud jpraynaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

I left few comments and suggestions below.

* more explicit parameter name for `DeleteCardanoTransactionProvider` method
* Simplify `TransactionPruner` doc
* Remove 'decorator' from the transaction pruner name
* Simplify some generics declaration
@Alenar Alenar temporarily deployed to testing-preview May 17, 2024 15:01 — with GitHub Actions Inactive
@Alenar Alenar temporarily deployed to testing-sanchonet May 17, 2024 15:01 — with GitHub Actions Inactive
Alenar added 2 commits May 17, 2024 17:01
* Mithril-aggregator from `0.5.7` to `0.5.8`
* Mithril-signer from `0.2.135` to `0.2.136`
* Mithril-persistence from `0.1.11` to `0.1.12`
@Alenar Alenar temporarily deployed to testing-preview May 17, 2024 15:08 — with GitHub Actions Inactive
@Alenar Alenar temporarily deployed to testing-sanchonet May 17, 2024 15:08 — with GitHub Actions Inactive
@Alenar Alenar merged commit 9104258 into main May 17, 2024
41 checks passed
@Alenar Alenar deleted the djo/1645/prune_ctx_in_signer branch May 17, 2024 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prune Cardano transactions stored on signer
3 participants