Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC 12 Implementation: Coordination layer groundwork #3744

Merged
merged 13 commits into from
Nov 27, 2023
Merged

Conversation

lukasz-zimnoch
Copy link
Member

@lukasz-zimnoch lukasz-zimnoch commented Nov 24, 2023

Refs: keep-network/tbtc-v2#737

Here we present the first part of the changes meant to implement RFC 12: Decentralized wallet coordination in the tBTC wallet client. This pull request focuses on the groundwork necessary to run the off-chain coordination layer.

Coordination layer orchestration

The node orchestrates the coordination layer upon startup. Specifically, it runs two separate goroutines:

  • Coordination window watch
  • Coordination result processor

Both goroutines are steered by the node's root context and communicate through a dedicated channel.

Coordination window watch

The coordination window watch goroutine is responsible for detecting coordination windows that occur every 900 blocks. Once a window is detected, it runs the window handler function and passes the window data as an argument.
The window watch process guarantees that the window handler is called only once for the given coordination window. Moreover, it also guarantees that the window handler is called for all coordination windows. This is achieved by calling each handler in a separate goroutine so the watch loop does not block for long and the chance of missing a coordination window signal is negligible.

The window handler function triggers the window processing logic. Specific steps of that logic are:

  1. Determine the list of wallets controlled by the given node
  2. Take a coordination executor for each wallet
  3. Use the coordination executors to run the coordination procedure for each wallet (in parallel)
  4. Collect coordination results and push them to the processing channel

Coordination executor

The coordination executor is a component that is responsible for running the coordination procedure for the given wallet and coordination window. It is designed to encapsulate the logic of the procedure (coordination seed, communication, and so on). It also ensures that only one instance of the procedure is executed at a time. The executor is also responsible for assembling the coordination procedure's result and reporting all coordination faults detected during execution.

The design of the coordination executor is inspired by the existing signing and DKG executor. It attempts to fit the coordination procedure's logic into the existing codebase in an elegant way.

Coordination result processor

The coordination result processor goroutine listens for incoming coordination results and triggers the result handler function in a separate goroutine to ensure all results are processed independently.

The result handler function triggers the result processing logic. Specific steps of that logic are:

  1. Record coordination faults reported in the result
  2. Detect the type of the action proposal being part of the result
  3. Fire the appropriate proposal handler

The proposal handlers are part of the existing codebase. They are responsible for the orchestration and execution of the proposed wallet actions.

Intersection with the existing chain-based coordination mechanism

The presented groundwork was built alongside the existing chain-based coordination mechanism. Some initial integration steps around data types were done. The existing mechanism will be gradually removed and replaced in the follow-up pull requests.

Next steps

The next steps (coarse-grained) on the way towards RFC 12 implementation are:

  • Implement coordination procedure logic (i.e. implement the coordinationExecutor.coordinate method)
  • Finalize coordination result processing (i.e. implement the processCoordinationResult function and refactor node's handlers appropriately)
  • Remove the existing chain-based mechanism (i.e. detach WalletCoordinator's events handlers and remove unnecessary code from chain.go)
  • Modify the SPV maintainter to not rely on WalletCoordinator's events during unproven transactions lookup

Here we add some basic primitives necessary to
implement the mechanism of decentralized wallet
coordination described in the RFC 12.
Here we add some code responsible for orchestration
of the wallet coordination layer.
Here we implement a coordination result processing
loop that receives results and directs them to
the right handler.
@lukasz-zimnoch lukasz-zimnoch changed the title RFC 12 Implementation? RFC 12 Implementation: Coordination layer Nov 24, 2023
@lukasz-zimnoch lukasz-zimnoch marked this pull request as draft November 24, 2023 09:49
@lukasz-zimnoch lukasz-zimnoch changed the title RFC 12 Implementation: Coordination layer RFC 12 Implementation: Coordination layer groundwork Nov 24, 2023
@lukasz-zimnoch lukasz-zimnoch self-assigned this Nov 24, 2023
Here we add two methods to the `coordinationWindow` type:

The `activePhaseEndBlock` denotes the end block of the active communication
phase and will be used to complete communication in the right moment.

The `endBlock` denotes the end block of the whole window and will be used
to complete the coordination procedure.
This is the first step to integrate the existing proposal structures with
the new mechanism. Here we are making them compliant with the
`coordinationProposal` interface and moving them out of the `chain.go` file
to indicate that they are no longer pure chain components.
@lukasz-zimnoch lukasz-zimnoch marked this pull request as ready for review November 24, 2023 14:15
Co-authored-by: Piotr Dyraga <piotr.dyraga@keep.network>
pkg/tbtc/coordination.go Show resolved Hide resolved
pkg/tbtc/node.go Show resolved Hide resolved
pkg/tbtc/node_test.go Show resolved Hide resolved
pkg/tbtc/deposit_sweep.go Show resolved Hide resolved
@pdyraga
Copy link
Member

pdyraga commented Nov 27, 2023

Did a high-level review, looks good to me. Leaving it in the hands of @tomaszslabon.

@lukasz-zimnoch
Copy link
Member Author

Did a high-level review, looks good to me. Leaving it in the hands of @tomaszslabon.

Thanks for reviewing @pdyraga !

@tomaszslabon tomaszslabon merged commit 35f9ceb into main Nov 27, 2023
29 checks passed
@tomaszslabon tomaszslabon deleted the rfc-12-impl branch November 27, 2023 13:22
tomaszslabon added a commit that referenced this pull request Nov 30, 2023
Refs: keep-network/tbtc-v2#737
Depends on: #3744

Here we present the second part of the changes meant to implement [RFC
12: Decentralized wallet
coordination](https://github.com/keep-network/tbtc-v2/blob/main/docs/rfc/rfc-12.adoc)
in the tBTC wallet client. This pull request focuses on the coordination
procedure.

On the code level, this is about implementing the
`coordinationExecutor.coordinate` function.

### Coordination seed

Calculation of the coordination seed was implemented in the
`coordinationExecutor.getSeed` function. This function computes the seed
for the given coordination window according to the RFC 12 specification:
```
coordination_seed = sha256(wallet_public_key_hash | safe_block_hash)
```
The computed value is used to initialize the RNG for random operations
executed as part of the coordination procedure. Those operations are:
- Leader selection
- Heartbeat execution 

### Coordination leader

Leader selection was implemented in the `coordinationExecutor.getLeader`
function. It uses the coordination seed to select the leader operator
from all operators backing the coordinated wallet. The exact algorithm
is:
1. Get the list of unique operators
2. Sort the list by address in the ascending order
3. Randomly shuffle the list using an RNG initialized with the
coordination seed
4. Pick the first operator from the shuffled list

### Actions checklist

The next step is determining which wallet actions should be checked and
possibly proposed during the coordination window. A list of such actions
is called **actions checklist**. Actions on the list are ordered by
priority, in descending order.

For example, if the list is: `[Redemption, DepositSweep, Heartbeat]`,
the leader must start by checking if redemption can be proposed. If so,
it should do it. Otherwise, it should check the deposit sweep. If
neither redemption nor deposit sweep can be proposed, the leader should
propose a heartbeat.

Actions checklist assembling was implemented in the
`coordinationExecutor.getActionsChecklist` function. The exact checklist
depends on the coordination window. The specific algorithm is:
1. Check the possibility of redemption on every window
2. Check the possibility of a deposit sweep, then moved funds sweep,
then moving funds, every 16th window
3. Draw a decision regarding the heartbeat proposal, with a 12.5% chance
of success
 
### Leader's routine

If the given operator finds itself to be a leader for the given
coordination window, it executes the leader's routine implemented in the
`coordinationExecutor.executeLeaderRoutine` function. It uses the
actions checklist to generate a proposal and broadcasts the proposal to
the followers, using the underlying broadcast channel. It completes the
coordination procedure returning the generated proposal as a result.

### Proposal generator

The leader uses the proposal generator function to generate a proposal
based on the actions checklist for the given window. The generator is
expected to return a proposal for the first action from the checklist
that is valid for the given wallet's state. If none of the actions are
valid, the generator returns a no-op proposal.

Implementation of the proposal generator is beyond the scope of this
pull request and will be handled in a follow-up PR. The plan is to use
the code from `pkg/maintainer/wallet` that is currently used by the
wallet maintainer bot.

### Follower's routine

If the given operator is not the leader for the given coordination
window, it executes the follower's routine implemented in the
`coordinationExecutor.executeFollowerRoutine` function. It listens for
incoming coordination messages and accepts the first message that:
- Is of the proper type
- Is not a self-message
- Comes from a sender with a confirmed wallet membership
- Refers to the currently processed coordination window
- Refers to the coordinated wallet
- Comes from the coordination leader designated for the given window
- Holds a proposal referring to an action from the checklist or a no-op
proposal

The operator-follower completes the coordination procedure returning the
received proposal as a result.

### Next steps

The next steps on the way towards RFC 12 implementation are:

- Implement proposal generation
- Finalize coordination result processing (i.e. implement the
`processCoordinationResult` function and refactor `node`'s handlers
appropriately)
- Remove the existing chain-based mechanism (i.e. detach
`WalletCoordinator`'s events handlers and remove unnecessary code from
`chain.go`)
- Modify the SPV maintainter to not rely on `WalletCoordinator`'s events
during unproven transactions lookup
@lukasz-zimnoch lukasz-zimnoch added this to the v2.0.0 milestone Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants