Merge pull request #2350 from input-output-hk/dlachaume/2329/document-client-incremental-cardano-db

Docs: update clients documentation for incremental Cardano DB

Author: dlachaume

docs/website/root/glossary.md

@@ -10,6 +10,10 @@ Below is a comprehensive list of definitions for some common terms used in the M
 
 A beacon represents a point of the blockchain for which a [Mithril certificate](#certificate) is created. It embeds the [epoch](#epoch) of the [Cardano network](#cardano-network) that is targeted, and either the block number or the [immutable file number](#immutable-file-number).
 
+## Cardano database
+
+A Cardano database snapshot is a signed archive of the blockchain state that can be used by [Mithril clients](#mithril-client) to restore a [Cardano full node](#cardano-node). It is uniquely identified by its fingerprint or digest which is part of the message signed by the [Mithril network](#mithril-network).
+
 ## Cardano network
 
 The Cardano network is a proof-of-stake blockchain platform that supports the ada cryptocurrency.
@@ -88,10 +92,6 @@ The Mithril multi-signature is an aggregate of [individual signatures](#individu
 
 > More information is available on the [protocol page](./mithril/advanced/mithril-protocol/protocol.md).
 
-## Snapshot
-
-A Mithril snapshot is a signed archive of the blockchain state that can be used by [Mithril clients](#mithril-client) to restore a [Cardano full node](#cardano-node). It is uniquely identified by its fingerprint or digest which is part of the message signed by the [Mithril network](#mithril-network).
-
 ## Stake distribution
 
 The Cardano stake distribution is the list of all the [stake pool operators'](#stake-pool-operator-spo) pool Id addresses and their associated share of the total stake of the [Cardano network](#cardano-network).

docs/website/root/manual/develop/nodes/mithril-client-library-wasm.md

@@ -10,10 +10,10 @@ Mithril client library WASM can be used by Javascript developers to use the Mith
 
 It is responsible for handling the different types of data certified by Mithril and available through a Mithril aggregator:
 
-- [**Snapshot**](../../../glossary.md#snapshot): list and get
-- [**Mithril stake distribution**](../../../glossary.md#stake-distribution): list and get
 - [**Cardano transaction**](../../../glossary.md#cardano-transaction): list and get snapshots, get proofs
 - [**Cardano stake distribution**](../../../glossary.md#stake-distribution): list, get and get by epoch
+- [**Cardano database**](../../../glossary.md#cardano-database): list and get
+- [**Mithril stake distribution**](../../../glossary.md#stake-distribution): list and get
 - [**Certificate**](../../../glossary.md#certificate): list, get, and chain validation.
 
 :::

docs/website/root/manual/develop/nodes/mithril-client-library.md

@@ -10,10 +10,10 @@ Mithril client library can be used by Rust developers to use the Mithril network
 
 It is responsible for handling the different types of data certified by Mithril and available through a Mithril aggregator:
 
-- [**Snapshot**](../../../glossary.md#snapshot): list, get, download the tarball, and record statistics
-- [**Mithril stake distribution**](../../../glossary.md#stake-distribution): list and get
 - [**Cardano transaction**](../../../glossary.md#cardano-transaction): list and get snapshots, get proofs
 - [**Cardano stake distribution**](../../../glossary.md#stake-distribution): list, get, and get by epoch
+- [**Cardano database**](../../../glossary.md#cardano-database): list, get, download the tarball, and record statistics
+- [**Mithril stake distribution**](../../../glossary.md#stake-distribution): list and get
 - [**Certificate**](../../../glossary.md#certificate): list, get, and chain validation.
 
 :::
@@ -64,34 +64,168 @@ Mithril client is an asynchronous library. You will need a runtime to execute yo
 
 ## Using the Mithril client library
 
-Below is a basic example of how to use most of the functions exposed by the Mithril client library:
+### Cardano transactions
+
+Here is a basic example of the code targetting the `release-preprod` network aggregator:
+
+```rust title="/src/main.rs"
+use mithril_client::{ClientBuilder, MessageBuilder, MithrilResult};
+
+#[tokio::main]
+async fn main() -> MithrilResult<()> {
+    const AGGREGATOR_ENDPOINT: &str =
+        "https://aggregator.release-preprod.api.mithril.network/aggregator";
+    const GENESIS_VERIFICATION_KEY: &str = "5b3132372c37332c3132342c3136312c362c3133372c3133312c3231332c3230372c3131372c3139382c38352c3137362c3139392c3136322c3234312c36382c3132332c3131392c3134352c31332c3233322c3234332c34392c3232392c322c3234392c3230352c3230352c33392c3233352c34345d";
+    let client =
+        ClientBuilder::aggregator(AGGREGATOR_ENDPOINT, GENESIS_VERIFICATION_KEY).build()?;
+
+    let transactions_hashes = [
+        "f9b5221b3ead45d46c0ecae6bee18a0746c5694d0285281cca1b651bce5f49a5",
+        "7769e8b78cc86890660ff5451c110b0a0d0413c8b8ebb17a64e017b4cd881777",
+    ];
+    let cardano_transaction_proof = client
+        .cardano_transaction()
+        .get_proofs(&transactions_hashes)
+        .await
+        .unwrap();
+
+    let verified_transactions = cardano_transaction_proof.verify().unwrap();
+
+    let certificate = client
+        .certificate()
+        .verify_chain(&cardano_transaction_proof.certificate_hash)
+        .await
+        .unwrap();
+
+    let message = MessageBuilder::new()
+        .compute_cardano_transactions_proofs_message(&certificate, &verified_transactions);
+    assert!(certificate.match_message(&message));
+
+    println!(
+        r###"Cardano transactions with hashes "'{}'" have been successfully certified by Mithril."###,
+        verified_transactions.certified_transactions().join(","),
+    );
+    if !cardano_transaction_proof
+        .non_certified_transactions
+        .is_empty()
+    {
+        println!(
+            r###"No proof could be computed for Cardano transactions with hashes "'{}'".
+
+            Mithril may not have signed those transactions yet, please try again later."###,
+            cardano_transaction_proof
+                .non_certified_transactions
+                .join(","),
+        );
+    }
+
+    Ok(())
+}
+```
+
+:::info
+
+An full example is available in the [Mithril repository](https://github.com/input-output-hk/mithril/tree/main/examples/client-cardano-transaction/src/main.rs). To run it, execute the following command:
+
+```bash
+cargo run -p client-cardano-transaction <TRANSACTIONS_HASHES>
+```
+
+or directly from the example crate directory:
+
+```bash
+cargo run
+```
+
+:::
+
+### Cardano stake distribution
+
+Here is a basic example of the code targetting the `release-preprod` network aggregator:
+
+```rust title="/src/main.rs"
+use mithril_client::{ClientBuilder, MessageBuilder, MithrilResult};
+
+#[tokio::main]
+async fn main() -> MithrilResult<()> {
+    const AGGREGATOR_ENDPOINT: &str =
+        "https://aggregator.release-preprod.api.mithril.network/aggregator";
+    const GENESIS_VERIFICATION_KEY: &str = "5b3132372c37332c3132342c3136312c362c3133372c3133312c3231332c3230372c3131372c3139382c38352c3137362c3139392c3136322c3234312c36382c3132332c3131392c3134352c31332c3233322c3234332c34392c3232392c322c3234392c3230352c3230352c33392c3233352c34345d";
+    let client =
+        ClientBuilder::aggregator(AGGREGATOR_ENDPOINT, GENESIS_VERIFICATION_KEY).build()?;
+
+    let cardano_stake_distributions = client.cardano_stake_distribution().list().await?;
+    let last_epoch = cardano_stake_distributions.first().unwrap().epoch;
+
+    let cardano_stake_distribution = client
+        .cardano_stake_distribution()
+        .get_by_epoch(last_epoch)
+        .await?
+        .unwrap();
+
+    let certificate = client
+        .certificate()
+        .verify_chain(&cardano_stake_distribution.certificate_hash)
+        .await?;
+
+    let message = MessageBuilder::new()
+        .compute_cardano_stake_distribution_message(&certificate, &cardano_stake_distribution)?;
+
+    assert!(certificate.match_message(&message));
+
+    Ok(())
+}
+```
+
+:::info
+
+An full example is available in the [Mithril repository](https://github.com/input-output-hk/mithril/tree/main/examples/client-cardano-stake-distribution/src/main.rs). To run it, execute the following command:
+
+```bash
+cargo run -p client-cardano-stake-distribution
+```
+
+or directly from the example crate directory:
+
+```bash
+cargo run
+```
+
+:::
+
+### Cardano database
+
+Here is a basic example of the code targetting the `release-preprod` network aggregator:
 
 ```rust title="/src/main.rs"
 use mithril_client::{ClientBuilder, MessageBuilder};
 use std::path::Path;
 
 #[tokio::main]
 async fn main() -> mithril_client::MithrilResult<()> {
-    let client = ClientBuilder::aggregator("YOUR_AGGREGATOR_ENDPOINT", "YOUR_GENESIS_VERIFICATION_KEY").build()?;
-
-    let snapshots = client.snapshot().list().await?;
+    const AGGREGATOR_ENDPOINT: &str =
+        "https://aggregator.release-preprod.api.mithril.network/aggregator";
+    const GENESIS_VERIFICATION_KEY: &str = "5b3132372c37332c3132342c3136312c362c3133372c3133312c3231332c3230372c3131372c3139382c38352c3137362c3139392c3136322c3234312c36382c3132332c3131392c3134352c31332c3233322c3234332c34392c3232392c322c3234392c3230352c3230352c33392c3233352c34345d";
+    let client =
+        ClientBuilder::aggregator(AGGREGATOR_ENDPOINT, GENESIS_VERIFICATION_KEY).build()?;
+    let snapshots = client.cardano_database().list().await?;
 
     let last_digest = snapshots.first().unwrap().digest.as_ref();
-    let snapshot = client.snapshot().get(last_digest).await?.unwrap();
+    let snapshot = client.cardano_database().get(last_digest).await?.unwrap();
 
     let certificate = client
         .certificate()
         .verify_chain(&snapshot.certificate_hash)
         .await?;
 
     // Note: the directory must already exist, and the user running this code must have read/write access to it.
-    let target_directory = Path::new("YOUR_TARGET_DIRECTORY");
+    let target_directory = Path::new(".");
     client
-        .snapshot()
+        .cardano_database()
         .download_unpack(&snapshot, target_directory)
         .await?;
 
-    if let Err(e) = client.snapshot().add_statistics(&snapshot).await {
+    if let Err(e) = client.cardano_database().add_statistics(&snapshot).await {
         println!("Could not increment snapshot download statistics: {:?}", e);
     }
 
@@ -108,10 +242,10 @@ async fn main() -> mithril_client::MithrilResult<()> {
 
 Snapshot download and certificate chain validation can take quite some time, even with a fast computer and network. We have implemented a feedback mechanism for them; more details are available in the [feedback sub-module](https://mithril.network/rust-doc/mithril_client/feedback/index.html).
 
-An example of implementation with the crate [indicatif](https://crates.io/crates/indicatif) is available in the [Mithril repository](https://github.com/input-output-hk/mithril/tree/main/examples/client-snapshot/src/main.rs). To run it, execute the following command:
+An example of implementation with the crate [indicatif](https://crates.io/crates/indicatif) is available in the [Mithril repository](https://github.com/input-output-hk/mithril/tree/main/examples/client-cardano-database/src/main.rs). To run it, execute the following command:
 
 ```bash
-cargo run -p client-snapshot
+cargo run -p client-cardano-database
 ```
 
 or directly from the example crate directory:
@@ -122,46 +256,122 @@ cargo run
 
 :::
 
-Here is a working example of the code using the configuration parameters of the `release-preprod` network:
+### Cardano database v2
+
+:::danger
+
+This feature is still unstable.
+
+To use it, you need to add the `unstable` feature in your project's `Cargo.toml` file.
+
+```
+mithril-client = { version = "0.11.X", features = ["fs", "unstable"] }
+```
+
+:::
+
+Below is a basic example using the new `CardanoDatabase` functions. Make sure the target aggregator signs `CardanoDatabase` incremental snapshot.
+
+:::tip
+
+You can verify that the aggregator signs **CardanoDatabase** by running the command below:
+
+```bash
+wget -q -O - YOUR_AGGREGATOR_ENDPOINT | jq '.capabilities.signed_entity_types | contains(["CardanoDatabase"])'
+```
+
+For example, with the aggregator on the `testing-preview` Mithril network:
+
+```bash
+wget -q -O - https://aggregator.testing-preview.api.mithril.network/aggregator | jq '.capabilities.signed_entity_types | contains(["CardanoDatabase"])'
+```
+
+:::
 
 ```rust title="/src/main.rs"
-use mithril_client::{ClientBuilder, MessageBuilder};
+use mithril_client::{
+    cardano_database_client::{DownloadUnpackOptions, ImmutableFileRange},
+    ClientBuilder, MessageBuilder,
+};
 use std::path::Path;
 
 #[tokio::main]
 async fn main() -> mithril_client::MithrilResult<()> {
-    let client = ClientBuilder::aggregator("https://aggregator.release-preprod.api.mithril.network/aggregator", "5b3132372c37332c3132342c3136312c362c3133372c3133312c3231332c3230372c3131372c3139382c38352c3137362c3139392c3136322c3234312c36382c3132332c3131392c3134352c31332c3233322c3234332c34392c3232392c322c3234392c3230352c3230352c33392c3233352c34345d").build()?;
+    const AGGREGATOR_ENDPOINT: &str =
+        "https://aggregator.testing-preview.api.mithril.network/aggregator";
+    const GENESIS_VERIFICATION_KEY: &str = "5b3132372c37332c3132342c3136312c362c3133372c3133312c3231332c3230372c3131372c3139382c38352c3137362c3139392c3136322c3234312c36382c3132332c3131392c3134352c31332c3233322c3234332c34392c3232392c322c3234392c3230352c3230352c33392c3233352c34345d";
+    let client =
+        ClientBuilder::aggregator(AGGREGATOR_ENDPOINT, GENESIS_VERIFICATION_KEY).build()?;
 
-    let snapshots = client.snapshot().list().await?;
+    let snapshots = client.cardano_database_v2().list().await?;
 
-    let last_digest = snapshots.first().unwrap().digest.as_ref();
-    let snapshot = client.snapshot().get(last_digest).await?.unwrap();
+    let latest_hash = snapshots.first().unwrap().hash.as_ref();
+    let snapshot = client
+        .cardano_database_v2()
+        .get(latest_hash)
+        .await?
+        .unwrap();
 
     let certificate = client
         .certificate()
         .verify_chain(&snapshot.certificate_hash)
         .await?;
 
+    let immutable_file_range = ImmutableFileRange::From(15000);
+    let download_unpack_options = DownloadUnpackOptions {
+        allow_override: true,
+        include_ancillary: false,
+        ..DownloadUnpackOptions::default()
+    };
+
     // Note: the directory must already exist, and the user running this code must have read/write access to it.
     let target_directory = Path::new(".");
     client
-        .snapshot()
-        .download_unpack(&snapshot, target_directory)
+        .cardano_database_v2()
+        .download_unpack(
+            &snapshot,
+            &immutable_file_range,
+            target_directory,
+            download_unpack_options,
+        )
         .await?;
 
-    if let Err(e) = client.snapshot().add_statistics(&snapshot).await {
-        println!("Could not increment snapshot download statistics: {:?}", e);
-    }
+    let merkle_proof = client
+        .cardano_database_v2()
+        .compute_merkle_proof(
+            &certificate,
+            &snapshot,
+            &immutable_file_range,
+            target_directory,
+        )
+        .await?;
+    merkle_proof.verify()?;
 
     let message = MessageBuilder::new()
-        .compute_snapshot_message(&certificate, target_directory)
+        .compute_cardano_database_message(&certificate, &merkle_proof)
         .await?;
     assert!(certificate.match_message(&message));
 
     Ok(())
 }
 ```
 
+:::info
+
+An full example is available in the [Mithril repository](https://github.com/input-output-hk/mithril/tree/main/examples/client-cardano-database-v2/src/main.rs). To run it, execute the following command:
+
+```bash
+cargo run -p client-cardano-database-v2
+```
+
+or directly from the example crate directory:
+
+```bash
+cargo run
+```
+
+:::
+
 :::tip
 
 You can read the complete [developer documentation](https://mithril.network/rust-doc/mithril_client/index.html).