Crate-level documentation on finality relays and relayers pallet (par…

…itytech#1773)

* started crate-level documentation on relays and relayers pallet

* fixes in finality and parachain finality relay docs

* fix links

* grammar fixes

docs/high-level-overview.md

@@ -41,7 +41,7 @@ proofs are called mandatory in the pallet and relayer pays no fee for such heade
 The pallet does not require all headers to be imported or provided. The relayer itself chooses which headers
 he wants to submit (with the exception of mandatory headers).
 
-More: [code](../modules/grandpa/).
+More: [pallet level documentation and code](../modules/grandpa/).
 
 ### Bridge Parachains Finality Pallet
 
@@ -60,7 +60,7 @@ The pallet may track multiple parachains at once and those parachains may use di
 parachain header decoding never happens at the pallet level. For maintaining the headers order, the pallet
 uses relay chain header number.
 
-More: [code](../modules/parachains/).
+More: [pallet level documentation and code](../modules/parachains/).
 
 ### Bridge Messages Pallet
 
@@ -91,14 +91,14 @@ pallet, in this case, depends on one of the finality pallets. The messages are X
 XCM executor to dispatch them on receival. You may find more info in [Polkadot <> Kusama Bridge](./polkadot-kusama-bridge-overview.md)
 document.
 
-More: [code](../modules/messages/).
+More: [pallet level documentation and code](../modules/messages/).
 
 ### Bridge Relayers Pallet
 
 The pallet is quite simple. It just registers relayer rewards and has an entrypoint to collect them. When
 the rewards are registered and the reward amount is configured outside of the pallet.
 
-More: [code](../modules/relayers/).
+More: [pallet level documentation and code](../modules/relayers/).
 
 ## Offchain Components
 

modules/relayers/README.md

@@ -0,0 +1,14 @@
+# Bridge Relayers Pallet
+
+The pallet serves as a storage for pending bridge relayer rewards. Any runtime component may register reward
+to some relayer for doing some useful job at some messages lane. Later, the relayer may claim its rewards
+using the `claim_rewards` call.
+
+The reward payment procedure is abstracted from the pallet code. One of possible implementations, is the
+[`PayLaneRewardFromAccount`](../../primitives/relayers/src/lib.rs), which just does a `Currency::transfer`
+call to relayer account from the relayer-rewards account, determined by the message lane id.
+
+We have two examples of how this pallet is used in production. Rewards are registered at the target chain to
+compensate fees of message delivery transactions (and linked finality delivery calls). At the source chain, rewards
+are registered during delivery confirmation transactions. You may find more information about that in the
+[Kusama <> Polkadot bridge](../../docs/polkadot-kusama-bridge-overview.md) documentation.

modules/shift-session-manager/README.md

@@ -0,0 +1,10 @@
+# Shift Session Manager Pallet
+
+**THIS PALLET IS NOT INTENDED TO BE USED IN PRODUCTION**
+
+The pallet does not provide any calls or runtime storage entries. It only provides implementation of the
+`pallet_session::SessionManager`. This implementation, starting from session `3` selects two thirds of initial
+validators and changes the set on every session. We are using it at our testnets ([Rialto](../../bin/rialto/) and
+[Millau](../../bin/millau/)) to be sure that the set changes every session. On well-known production chains
+(like Kusama and Polkadot) the alternative is the set of [nPoS](https://research.web3.foundation/en/latest/polkadot/NPoS/index.html)
+pallets, which selects validators, based on their nominations.

relays/finality/README.md

@@ -0,0 +1,58 @@
+# GRANDPA Finality Relay
+
+The finality relay is able to work with different finality engines. In the modern Substrate world they are GRANDPA
+and BEEFY. Let's talk about GRANDPA here, because BEEFY relay and bridge BEEFY pallet are in development.
+
+In general, the relay works as follows: it connects to the source and target chain. The source chain must have the 
+[GRANDPA gadget](https://github.com/paritytech/finality-grandpa) running (so it can't be a parachain). The target
+chain must have the [bridge GRANDPA pallet](../../modules/grandpa/) deployed at its runtime. The relay subscribes
+to the GRANDPA finality notifications at the source chain and when the new justification is received, it is submitted
+to the pallet at the target chain.
+
+Apart from that, the relay is watching for every source header that is missing at target. If it finds the missing
+mandatory header (header that is changing the current GRANDPA validators set), it submits the justification for
+this header. The case when the source node can't return the mandatory justification is considered a fatal error,
+because the pallet can't proceed without it.
+
+More: [GRANDPA Finality Relay Sequence Diagram](../../docs/grandpa-finality-relay.html).
+
+## How to Use the Finality Relay
+
+The most important trait is the [`FinalitySyncPipeline`](./src/lib.rs), which defines the basic primitives of the
+source chain (like block hash and number) and the type of finality proof (GRANDPA jusitfication or MMR proof). Once
+that is defined, there are two other traits - [`SourceClient`](./src/finality_loop.rs) and
+[`TarggetClient`](./src/finality_loop.rs).
+
+The `SourceClient` represents the Substrate node client that connects to the source chain. The client need to
+be able to return the best finalized header number, finalized header and its finality proof and the stream of
+finality proofs.
+
+The `TargetClient` implementation must be able to craft finality delivery transaction and submit it to the target
+node. The transaction is then tracked by the relay until it is mined and finalized.
+
+The main entrypoint for the crate is the [`run` function](./src/finality_loop.rs), which takes source and target
+clients and [`FinalitySyncParams`](./src/finality_loop.rs) parameters. The most imporant parameter is the
+`only_mandatory_headers` - it it is set to `true`, the relay will only submit mandatory headers. Since transactions
+with mandatory headers are fee-free, the cost of running such relay is zero (in terms of fees).
+
+## Finality Relay Metrics
+
+Finality relay provides several metrics. Metrics names depend on names of source and target chains. The list below
+shows metrics names for Rialto (source chain) to Millau (target chain) finality relay. For other chains, simply
+change chain names. So the metrics are:
+
+- `Rialto_to_Millau_Sync_best_source_block_number` - returns best finalized source chain (Rialto) block number, known
+  to the relay. If relay is running in [on-demand mode](../bin-substrate/src/cli/relay_headers_and_messages/), the
+  number may not match (it may be far behind) the actual best finalized number;
+
+- `Rialto_to_Millau_Sync_best_source_at_target_block_number` - returns best finalized source chain (Rialto) block
+  number that is known to the bridge GRANDPA pallet at the target chain.
+
+- `Rialto_to_Millau_Sync_is_source_and_source_at_target_using_different_forks` - if this metrics is set to `1`, then
+  the best source chain header, known to the target chain doesn't match the same-number-header at the source chain.
+  It means that the GRANDPA validators set has crafted the duplicate justification and it has been submitted to the
+  target chain. Normally (if majority of validators are honest and if you're running finality relay without large
+  breaks) this shall not happen and the metric will have `0` value.
+
+If relay operates properly, you should see that the `Rialto_to_Millau_Sync_best_source_at_target_block_number`
+tries to reach the `Rialto_to_Millau_Sync_best_source_block_number`. And the latter one always increases.

relays/parachains/README.md

@@ -0,0 +1,49 @@
+# Parachains Finality Relay
+
+The parachains finality relay works with two chains - source relay chain and target chain (which may be standalone
+chain, relay chain or a parachain). The source chain must have the 
+[`paras` pallet](https://github.com/paritytech/polkadot/tree/master/runtime/parachains/src/paras) deployed at its
+runtime. The target chain must have the [bridge parachains pallet](../../modules/parachains/) deployed at its runtime.
+
+The relay is configured to submit heads of one or several parachains. It pokes source chain periodically and compares
+parachain heads that are known to the source relay chain to heads at the target chain. If there are new heads,
+the relay submits them to the target chain.
+
+More: [Parachains Finality Relay Sequence Diagram](../../docs/parachains-finality-relay.html).
+
+## How to Use the Parachains Finality Relay
+
+There are only two traits that need to be implemented. The [`SourceChain`](./src/parachains_loop.rs) implementation
+is supposed to connect to the source chain node. It must be able to read parachain heads from the `Heads` map of
+the [`paras` pallet](https://github.com/paritytech/polkadot/tree/master/runtime/parachains/src/paras) pallet.
+It also must create storage proofs of `Heads` map entries, when required.
+
+The [`TargetChain`](./src/parachains_loop.rs) implementation connects to the target chain node. It must be able
+to return the best known head of given parachain. When required, it must be able to craft and submit parachains
+finality delivery transaction to the target node.
+
+The main entrypoint for the crate is the [`run` function](./src/parachains_loop.rs), which takes source and target
+clients and [`ParachainSyncParams`](./src/parachains_loop.rs) parameters. The most imporant parameter is the
+`parachains` - it it the set of parachains, which relay tracks and updates. The other important parameter that
+may affect the relay operational costs is the `strategy`. If it is set to `Any`, then the finality delivery
+transaction is submitted if at least one of tracked parachain heads is updated. The other option is `All`. Then
+the relay waits until all tracked parachain heads are updated and submits them all in a single finality delivery
+transaction.
+
+## Parachain Finality Relay Metrics
+
+Every parachain in Polkadot is identified by the 32-bit number. All metrics, exposed by the parachains finality
+relay have the `parachain` label, which is set to the parachain id. And the metrics are prefixed with the prefix,
+that depends on the name of the source relay and target chains. The list below shows metrics names for
+Rialto (source relay chain) to Millau (target chain) parachains finality relay. For other chains, simply
+change chain names. So the metrics are:
+
+- `Rialto_to_Millau_Parachains_best_parachain_block_number_at_source` - returns best known parachain block
+   number, registered in the `paras` pallet at the source relay chain (Rialto in our example);
+
+- `Rialto_to_Millau_Parachains_best_parachain_block_number_at_target` - returns best known parachain block
+   number, registered in the bridge parachains pallet at the target chain (Millau in our example).
+
+If relay operates properly, you should see that the `Rialto_to_Millau_Parachains_best_parachain_block_number_at_target`
+tries to reach the `Rialto_to_Millau_Parachains_best_parachain_block_number_at_source`. And the latter one
+always increases.