docs: tss block signing proposal

platform-sdk/docs/proposals/TSS-Block-Signing/tss-block-signing.md

@@ -0,0 +1,319 @@
+# TSS Block Signing
+
+---
+
+## Summary
+
+This proposal builds on the `TSS-Ledger-ID` proposal by extending the `TssBaseService` with the API and
+implementation for requesting ledger signatures and registering consumers of ledger signatures once they are produced.
+
+|      Metadata      |                  Entities                  |
+|--------------------|--------------------------------------------|
+| Designers          | Edward Wertz, Richard Bair, Michael Tinker |
+| Functional Impacts | Services                                   |
+| Related Proposals  | TSS-Ledger-ID                              |
+| HIPS               | HIP-1, HIP-2,                              |
+
+---
+
+## Purpose and Context
+
+This proposal is the 4th of 6 proposals that deliver the full Threshold Signature Scheme (TSS) capability based on
+[BLS](https://en.wikipedia.org/wiki/BLS_digital_signature). This TSS capability creates a network level
+private/public key pair where the public key is called the `ledger id` and is known by everyone, and the ledger's
+private key is not known by anyone. Nodes in the network have been given special private/public key pairs, called
+`shares`, whose signatures on a message can be aggregated to produce a `ledger signature` equivalent to the signature
+generated by the ledger's private key signing the same message. Ledger signatures are verifiable by the ledger id. The
+value of our chosen TSS scheme is that a network of nodes is able to transfer the ability to generate ledger
+signatures to another network of nodes without revealing the ledger private key.
+
+TSS Proposals:
+
+1. The `TSS-Library` proposal contains the cryptographic primitives and algorithms needed to implement TSS.
+2. The `TSS-Roster` proposal introduces the data structure of a consensus `Roster` to the platform.
+3. The `TSS-Ledger-ID` proposal introduces a ledger id for existing networks.
+4. This proposal (`TSS-Block-Signing`) introduces the methodology for signing blocks.
+5. The `TSS-Ledger-ID-Updates` proposal covers the process of resetting and transplanting ledger ids between networks.
+6. The `TSS-Ledger-New-Networks` proposal covers the process of setting up a new network with a ledger id.
+
+This proposal, `TSS-Block-Signing` covers the following:
+
+- `TssBaseService` API
+  - Add a method for requesting ledger signatures on a message.
+  - Add a method for registering consumers of ledger signatures.
+- `TssBaseService` Implementation
+  - Each node gossips a `TssShareSignatureTransaction` for each share signature it produces.
+  - A threshold number of `TssShareSignatureTransaction` are aggregated to produce the ledger signature.
+
+### Goals
+
+The following are goals of this proposal:
+
+1. That existing networks are able to create ledger signatures if the active roster has key material that can
+   recover the network's ledger id.  (See `TSS-Ledger-ID`)
+
+### Non-Goals
+
+The following is not part of the scope of this proposal:
+
+1. The ability to create ledger signatures for new networks.
+2. Integrating this capability into the broader block-stream effort.
+
+### Dependencies, Interactions, and Implications
+
+Dependencies on `TSS-Library`:
+
+- Upon availability of the implementation:
+  - This proposal can be tested with a pre-rendered set of shares, ledger private key, and ledger id.
+
+Dependencies on `TSS-Ledger-ID`:
+
+- Upon availability of the API:
+  - The `TssBaseService` API can be extended by this proposal.
+- Upon availability of the implementation:
+  - This proposal's full behavior can be tested in integration tests.
+
+Impacts to the Services Team:
+
+- The ability to sign blocks with TSS is blocked until this proposal delivers its capability.
+
+Implication Of Completion:
+
+- Signing blocks and construction of block proofs with TSS signatures is unblocked.
+
+### Requirements
+
+The core requirements governing the TSS proposals are stated in `TSS-Ledger-ID`.
+
+The following Block Signing specific requirements come from the Block Stream effort:
+
+1. A block is considered signed if its root hash has a ledger signature or a following block's root hash has a
+   ledger signature.
+2. TSS ledger signatures are best-effort with low probability of failure.
+3. The block proofs produced by different nodes for the same block do not need to be identical.
+4. Nodes attempt to sign every block.
+5. The number of consecutive blocks that are not signed is minimized.
+
+The rationale for these requirements is that we can
+construct [Merkle Proofs](https://ethereum.org/en/developers/tutorials/merkle-proofs-for-offline-data-integrity/) to
+extend the signature on Block N to Blocks N-1, N-2, ... all the way back to Block 1. Each block contains the root
+hash of the previous block as its first element with 1 internal merkle node between the root hash of a block and
+the root hash of the previous block. The length of a Merkle Proof extending the signature from Block N to Block
+(N-Y) is 3Y+1 hashes.
+
+To construct a TSS ledger signature, more than 1/2 the shares must sign the same message. These share signatures
+are then aggregated to produce the ledger signature. Shares are proportional representations of weight. In order for
+consensus to advance, 2/3 of weight must be participating in consensus. It is highly improbable that consensus can
+advance with 2/3 weight without the network being able to generate ledger signatures with more than 1/2 number of
+shares.
+
+If we wanted a guarantee of generating a ledger signature on every block, we would need to add state that
+captures the accumulation of share signatures after consensus. If we can tolerate the possibility of failing to
+construct a ledger signature, we can avoid the introduction of state and process the share signature transactions
+pre-consensus. This allows us to construct signatures faster during healthy execution.
+
+### Design Decisions
+
+No Guaranteed Ledger Signature:
+
+- `TssShareSignatureTransaction` are handled pre-consensus in the pre-handle phase of transaction handling
+  - this is to allow the creation of signatures as fast as possible.
+- The `TssShareSignatureTransaction` are not stored in the state.
+
+This decision (coming from the Block Stream design) is to have faster signatures handled in pre-consensus and
+cover any missed signatures on blocks with Merkle Proofs using signatures from later blocks.
+
+#### Alternatives Considered
+
+Guaranteed Ledger Signature Generation:
+
+- requires storing share signatures in the state.
+- requires processing share signatures after consensus, introducing a delay in block signing.
+
+This was not selected for the following reasons:
+
+1. block proofs do not need to be identical between nodes.
+2. the probability of failure to sign a block is low.
+3. speed in signing a block matters since blocks are not published without signatures once they are signable.
+
+## Changes
+
+The changes are presented in the following order:
+
+1. Public API
+2. Component Architecture
+3. Core Behaviors
+4. Configuration
+5. Metrics
+6. Performance
+
+### Public API
+
+#### Services
+
+The `TssBaseService` is extended with the following API:
+
+```Java
+/**
+ * The TssBaseService will attempt to generate TSS key material for any set candidate roster, giving it a ledger id and
+ * the ability to generate ledger signatures that can be verified by the ledger id.  Once the candidate roster has
+ * received its full TSS key material, it can be made available for adoption by the platform.
+ * </p>
+ * The TssBaseService will also attempt to generate ledger signatures by aggregating share signatures produced by
+ * calling {@link #requestLedgerSignature(byte[])}.
+ *
+ */
+public interface TssBaseService {
+
+    ...
+
+    /**
+     * Requests a ledger signature on a message.  The ledger signature is computed asynchronously and returned
+     * to all consumers that have been registered through {@link #registerLedgerSignatureConsumer(Consumer)}.
+     *
+     * @param message The message bytes to be signed by the ledger.
+     */
+    void requestLedgerSignature(@Nonnull byte[] message);
+
+    /**
+     * Registers a consumer of the message and the ledger signature on the message.
+     *
+     * @param consumer the consumer of the message and its ledger signature.
+     */
+    void registerLedgerSignatureConsumer(@Nonnull final BiConsumer<byte[], byte[]> consumer);
+
+    /**
+     * Unregisters a consumer of the message and the ledger signature.  The input object  reference must match by
+     * object equality to the registered consumer that is being unregistered.
+     *
+     * @param consumer the consumer to unregister.
+     */
+    void unRegisterLedgerSignatureConsumer(@Nonnull final BiConsumer<byte[], byte[]> consumer);
+}
+```
+
+#### System Transactions
+
+The `TssShareSignatureTransaction` is a new system transaction that is only generated by the `TssBaseService` and
+not user generated. They should be gossiped with the highest priority and not be slowed by long queues of user
+transactions waiting to be gossiped. It is critical that we minimize the time it takes to create ledger signatures
+on blocks.
+
+It may be possible that a node is aggregating share signature transactions from multiple ledger signature requests.
+In addition to the `share_signature`, the `roster_hash` and `share_index` are needed to identify the share public
+key to use for validation. The `message_hash` is needed to identify the message that was signed. A ledger signature
+can be produced by a threshold number of `share_signatures` for the same message. The threshold value for
+aggregation is `(TS +2)/2` using integer division where `TS` is the total number of shares in the roster indicated by
+the `roster_hash`. See the `TSS-Ledger-ID` proposal for more information on the threshold value.
+
+```protobuf
+message TssShareSignatureTransaction {
+  /**
+   * The hash of the roster containing the node whose share produced the share signature.
+   */
+  bytes roster_hash = 1;
+
+  /**
+   * The index of the share that produced this share signature.
+   */
+  uint64 share_index = 2;
+
+  /**
+   * The hash of the message that was signed by the share.
+   */
+  bytes message_hash = 3;
+
+  /**
+   * The share signature produced by the share.
+   */
+  bytes share_signature = 4;
+}
+```
+
+### Architecture and/or Components
+
+In addition to the extension of the `TssBaseService` interface, the service's `TransactionHandler` is extended to
+handle `TssShareSignatureTransaction` in the `pre-handle` phase.
+
+### Core Behaviors
+
+Thread Execution:
+
+* All code related to requesting a ledger signature,
+  generating share signature transactions, and handling share signature transactions do not modify the state and
+  should not execute on the `transaction handling thread`.
+* As soon as the ledger signature for a message can be constructed, all registered consumers are notified with the
+  message hash and the ledger signature. If there are multiple consumers registered, whether they are given the
+  signature serially or in parallel makes no difference as the data is immutable and consumption is expected to be
+  idempotent.
+
+Ledger Signature Generation:
+
+* The storage of `TssShareSignatureTransaction` is only in memory and expires after a configured amount of time.
+* Should a node be reset in the middle of the collection of share signatures, most signatures will be recoverable
+  through PCES replay.
+* If through restart or reconnect, a node is not able to collect enough share signatures to produce a ledger signature,
+  the accumulated share signatures will be thrown away after their holding time expires. No notification is generated
+  about the failure to produce a ledger signature.
+  * This infrequent failure to produce a ledger signature is expected to be rare and tolerated by ledger signature
+    consumers.
+  * Failures will be logged at an `INFO` level and metrics will be kept to track their occurrence.
+
+### Configuration
+
+New Configuration:
+
+* `tss.signatureLivenessPeriod` - The amount of time a share signature is held in memory before being discarded.
+  * Default is 5 minutes.
+* `tss.ledgerSignatureFailureThreshold` - The number of consecutive failures to produce a ledger signature before
+  logging an error.
+  * Default is 2.
+
+### Metrics
+
+These are new TSS metrics:
+
+1. The time between the request of a ledger signature and the generation of it.
+2. The number of consecutive failures to generate a ledger signature.
+
+### Performance
+
+All new code execution is happening off the transaction handling thread. This should have no impact on consensus
+node operations prior to the production of blocks in the blockstream.
+
+If blocks in the block stream are being produced without block proofs, waiting for a block proof before sending the
+next block will introduce a delay in releasing blocks to the block stream.
+
+---
+
+## Test Plan
+
+### Unit Tests
+
+Standard Unit Test Scenarios:
+
+* Ledger signature request and generation of `ShareSignatureTransaction`s.
+* `ShareSignatureTransaction` collection, and aggregation into a ledger signature.
+
+### Integration Tests
+
+At this level, no integration tests are needed.
+
+It is expected that when block signing is used to create block proofs on blocks in the block stream, this is
+sufficient for integration tests.
+
+### Performance Tests
+
+The standard release performance tests are sufficient. No additional performance tests are needed as all execution
+is happening off of the transaction handling thread.
+
+---
+
+## Implementation and Delivery Plan
+
+All code paths are dependent on input to generate load or impact on the system. While no other service is making
+the request of a ledger signature, the deployment of incrementally developed code for this proposal will have
+no impact on delivered software.
+
+Calls to request the generation of a ledger id can be delivered with the same release that completes the development
+of this proposal.