Skip to content

Remove speculative shielded context #459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Remove speculative shielded context #459

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 18 additions & 10 deletions packages/docs/pages/integrating-with-namada/sdk/shielded-sync.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,26 +1,33 @@
# Shielded Sync

Before performing MASP operations such as querying shielded balances, you will need to sync the local shielded wallet with the chain (this
is equivalent to running the [`namadac shielded-sync`](../../users/shielded-accounts/shielded-sync.mdx) command when using the CLI).
Before performing MASP operations such as querying shielded balances, you will
need to sync the local shielded wallet with the chain (the
[CLI](../../users/shielded-accounts/shielded-sync.mdx) does this automatically
for the involved commands).

Syncing the shielded wallet directly from the chain is performance-intensive, so it's recommended to sync using a running
instance of the [`namada-masp-indexer`](https://github.com/anoma/namada-masp-indexer).
Syncing the shielded wallet directly from the chain is performance-intensive, so
it's recommended to sync using a running instance of the
[`namada-masp-indexer`](https://github.com/anoma/namada-masp-indexer).

Instances of shielded wallets are not used directly, but are wrapped in a `shielded context` which adds capabilities
to a shielded wallet for querying nodes and performing I/O.
Instances of shielded wallets are not used directly, but are wrapped in a
`shielded context` which adds capabilities to a shielded wallet for querying
nodes and performing I/O.

### Example code

This example demonstrates how to perform a shielded-sync from a masp-indexer endpoint, and save the synced context to disk.
This example demonstrates how to perform a shielded-sync from a masp-indexer
endpoint, and save the synced context to disk.

Add the following dependencies to your `Cargo.toml`:

```rust
reqwest = "0.11.4"
kdam = "0.5.2"
```

Then, the following function will sync the shielded wallet from a given chain context and save it to the directory
provided when [creating the context](./context.mdx).
Then, the following function will sync the shielded wallet from a given chain
context and save it to the directory provided when
[creating the context](./context.mdx).

```rust
use std::time::Duration;
Expand Down Expand Up @@ -115,4 +122,5 @@ async fn shielded_sync(sdk: impl Namada) {
}
```

In the above example, one would replace the URL in the `endpoint` with the proper URL for the desired MASP indexer.
In the above example, one would replace the URL in the `endpoint` with the
proper URL for the desired MASP indexer.
121 changes: 72 additions & 49 deletions packages/docs/pages/users/shielded-accounts/shielded-sync.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,84 +1,107 @@
import { Callout } from 'nextra-theme-docs'
import { Callout } from "nextra-theme-docs";

# Shielded Sync

In order to interact with the MASP, users need a local copy of all of the notes. This is
necessary to create new notes with the correct zk-proofs. Additionally, it is the only way
for a client to find out which notes are owned by the user, which is necessary for calculating balances.
In order to interact with the MASP, users need a local copy of all of the notes.
This is necessary to create new notes with the correct zk-proofs. Additionally,
it is the only way for a client to find out which notes are owned by the user,
which is necessary for calculating balances.

Shielded sync is the command that downloads and processes a local view of the MASP (up to
a given block height) and then stores it locally in a `shielded wallet`. This command must be
run periodically to stay in sync with the tip of the chain.
Shielded sync is the command that downloads and processes a local view of the
MASP (up to a given block height) and then stores it locally in a
`shielded wallet`.

## Running shielded sync

The simplest way to run shielded sync is via the command:
```bash copy
namadac shielded-sync
```
This assumes several things by default. It will look into the user's wallet and import any MASP
keys into the shielded wallet. These will be the keys used to attempt to decrypt the downloaded
MASP notes. Additional keys can be provided via:
The shielded sync command is invoked automatically by the cli commands that
require it, namely:

- `transfer`
- `unshield`
- `ibc-transfer` (when the source is shielded)
- `balance` (when the owner is shielded)
- `estimate-shielding-rewards`

This assumes several things by default. It will look into the user's wallet and
import any MASP keys into the shielded wallet. These will be the keys used to
attempt to decrypt the downloaded MASP notes. Additional keys can be provided
via:

```bash copy
namadac shielded-sync --spending-keys $SPENDING_KEY
namadac <$COMMAND> <$COMMAND_ARGS> --spending-keys $SPENDING_KEY
```

and / or

```bash copy
namadac shielded-sync --viewing-keys $VIEWING_KEY
namadac <$COMMAND> <$COMMAND_ARGS> --viewing-keys $VIEWING_KEY
```

<Callout>
The first time you wish to check the balance of a spending/viewing key, you must provide it to the shielded wallet with
the `--spending-keys|--viewing-keys` argument. On subsequent runs, you can simply use `namadac shielded-sync`, as these
keys will be stored in the shielded wallet.
The first time you wish to check the balance of a spending/viewing key, you
must provide it to the shielded wallet with the
`--spending-keys|--viewing-keys` argument. On subsequent runs this won't be
required, as these keys will be stored in the shielded wallet.
</Callout>

Shielded sync also assumes by default that the user wishes to sync to the tip of the chain. If this is not
the case, a maximal height to sync to can be provided:
Shielded sync also assumes by default that the user wishes to sync to the tip of
the chain. If this is not the case, a maximal height to sync to can be provided:

```bash copy
namadac shielded-sync --to-height $BLOCKHEIGHT
namadac <$COMMAND> <$COMMAND_ARGS> --to-height $BLOCKHEIGHT
```

## Interrupting

If the shielded wallet is sufficiently behind the tip of the chain, the amount of data to be downloaded
and the computational load of processing it can be cumbersome. This is one reason why a user might
wish to specify a maximal height to which to sync.
If the shielded wallet is sufficiently behind the tip of the chain, the amount
of data to be downloaded and the computational load of processing it can be
cumbersome. This is one reason why a user might wish to specify a maximal height
to which to sync.

Additionally, the user can interrupt shielded sync at anytime by pressing `CTRL-C` or similar stop
command. Shielded sync will receive this signal and save its work before stopping. This avoids
losing work and can be resumed the next time shielded sync is run.
Additionally, the user can interrupt shielded sync at anytime by pressing
`CTRL-C` or similar stop command. Shielded sync will receive this signal and
save its work before stopping. This avoids losing work and can be resumed the
next time shielded sync is run.

## Using a MASP indexer

A MASP indexer can do a lot of the processing of MASP data and make it available to users via a
web server that can handle large downloads and many requests. This is in contrast to syncing
by calling the RPC endpoints of node operators.
A MASP indexer can do a lot of the processing of MASP data and make it available
to users via a web server that can handle large downloads and many requests.
This is in contrast to syncing by calling the RPC endpoints of node operators.

Thus, the preferred and most efficient way to run shielded sync is by using an
indexer:

Thus, the preferred and most efficient way to run shielded sync is by using an indexer:
```bash copy
namadac shielded-sync --with-indexer $INDEXER_API_URL
namadac <$COMMAND> <$COMMAND_ARGS> --with-indexer $INDEXER_API_URL
```
The api endpoint `$INDEXER_API_URL` will be a url to the indexer ending with `/api/v1`.

Current publicly available MASP indexers can be discovered [here for mainnet](https://namada.community/infra/masp-indexers).
In general, all infratsructure for both mainnets and testnets - including testnet MASP indexers - can be found in [this github repo](https://github.com/Luminara-Hub/namada-ecosystem/tree/main/user-and-dev-tools).
The api endpoint `$INDEXER_API_URL` will be a url to the indexer ending with
`/api/v1`.

Current publicly available MASP indexers can be discovered
[here for mainnet](https://namada.community/infra/masp-indexers). In general,
all infrasructure for both mainnets and testnets - including testnet MASP
indexers - can be found in
[this github repo](https://github.com/Luminara-Hub/namada-ecosystem/tree/main/user-and-dev-tools).

## Recovering from data corruption

If, for some reason, the shielded wallet gets corrupted and shielded sync cannot function correctly,
a user can try to remove all local data and sync again from scratch. The shielded wallet is stored
in the user's chain directory and the generated files have the following names:
* `shielded.dat`
* `shielded.tmp`
* `speculative_shielded.dat`
* `shielded_sync.cache`
* `shielded_sync.cache.tmp`
If, for some reason, the shielded wallet gets corrupted and shielded sync cannot
function correctly, a user can try to remove all local data and sync again from
scratch. The shielded wallet is stored in the user's chain directory and the
generated files have the following names:

- `shielded.dat`
- `shielded.tmp`
- `shielded_sync.cache`
- `shielded_sync.cache.tmp`

If these files are deleted, shielded sync can be run in order to re-sync from scratch.
If these files are deleted, shielded sync can be run in order to re-sync from
scratch.

<Callout>
Removing or modifying these files is not recommended and should only be done as a last resort.
Also note that the user will need to provided the necessary viewing and spending keys to
shielded sync again.
</Callout>
Removing or modifying these files is not recommended and should only be done
as a last resort. Also note that the user will need to provided the necessary
viewing and spending keys to shielded sync again.
</Callout>
56 changes: 33 additions & 23 deletions packages/docs/pages/users/shielded-accounts/shielding.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,34 +1,47 @@
import { Steps } from 'nextra-theme-docs'
import { Callout } from 'nextra-theme-docs'
import { Steps } from "nextra-theme-docs";
import { Callout } from "nextra-theme-docs";

# Shielding Assets

You can shield assets in the MASP by making a transfer from a transparent address to a shielded address. This is known
as a __shielding transfer__.
You can shield assets in the MASP by making a transfer from a transparent
address to a shielded address. This is known as a **shielding transfer**.

Assets can also be sent directly to a shielded address through IBC -- see the section on [Shielded IBC](../../users/ibc/shielded-ibc.mdx) for details.
Assets can also be sent directly to a shielded address through IBC -- see the
section on [Shielded IBC](../../users/ibc/shielded-ibc.mdx) for details.

## Making a shielding transfer

To conduct a shielding transfer, the user must first be in possession of a transparent account with some token balance.
To conduct a shielding transfer, the user must first be in possession of a
transparent account with some token balance.

<Steps>

### Generate a spending key

See [Shielded Key Management](./masp-keys.mdx) for details on how to do this.

### Derive a new shielded address (aka: payment address)
You can (and should) derive multiple shielded addresses for the same spending key.

You can (and should) derive multiple shielded addresses for the same spending
key.

### (If needed) submit `reveal-pk` transaction for sending address
You can shield from either an [implicit](../transparent-accounts/implicit-accounts.mdx) or [established](../transparent-accounts/established-accounts.mdx) account. If
shielding from an implicit account, your account's public key must first be revealed on-chain. This only needs to be done once per account:

You can shield from either an
[implicit](../transparent-accounts/implicit-accounts.mdx) or
[established](../transparent-accounts/established-accounts.mdx) account. If
shielding from an implicit account, your account's public key must first be
revealed on-chain. This only needs to be done once per account:

```bash copy
namadac reveal-pk --public-key $IMPLICIT_ACCOUNT_ALIAS
```

### Send your shielding transfer
Use the `shield` command to make a shielding transfer from a source transparent (`tnam`) address to a target shielded (`znam`) address:

Use the `shield` command to make a shielding transfer from a source transparent
(`tnam`) address to a target shielded (`znam`) address:

```bash copy
namadac shield \
--source $TRANSPARENT_ACCOUNT \
Expand All @@ -38,24 +51,21 @@ namadac shield \
```

</Steps>
<Callout>
Note: Shielding assets, even using IBC, is an _optimistic operation_. This means that the shielded wallet will assume
that the transaction succeeded and store the state changes. When this happens, the shielded wallet is said to be in a
_speculative state_. This occurs even if the transaction ultimately fails.

To move out of a _speculative state_, run [shielded sync](shielded-sync.mdx). This is especially important if a shielding transaction failed as
the shielded wallet's view of the MASP pool has now diverged from the MASP pool on the chain.
</Callout>

### Viewing your balance
To view the up-to-date balance of your spending key (or viewing key), you must first run the `shielded-sync` command to
sync the local shielded wallet with any MASP notes owned by your spending/viewing key.
See how to do this [here](shielded-sync.mdx).

To view the up-to-date balance of your spending key (or viewing key), you must
first run the `shielded-sync` command to sync the local shielded wallet with any
MASP notes owned by your spending/viewing key. See how to do this
[here](shielded-sync.mdx).

After the sync has completed, check your balance with:

```bash copy
namadac balance --owner $SPENDING_KEY --token $TOKEN
```
(A viewing key can also be provided here instead of `$SPENDING_KEY`)

This will display the combined balance of all shielded addresses associated with that spending/viewing key.
(A viewing key can also be provided here instead of `$SPENDING_KEY`)

This will display the combined balance of all shielded addresses associated with
that spending/viewing key.
Loading