Skip to content

perf: add batch verification into txhandler #4549

@algorandskiy

Description

@algorandskiy

Implement a batch verifier in txhandler.
Idea: implement a new worker pool that batches work for verifier.

The purpose of this project is to take advantage of faster group verification of the signatures implemented here:
#3031

--------------- Notes from Shant---------------------------
This is being implemented in the following PRs:
#4614
This PR returns the indexes of the failed signatures from Batch Verify.

#4621
This PR delivers two main features:

  1. A method to batch the verifications of incoming transactions
  2. Streaming capabilities of transactions to the verifier
    Why (1):
    Batching transactions are more efficient to verify because of the cryptographic efficiency in doing so. It is used for verification of txns in the block and the Msig. With this work, it will also be possible for incoming txns.

Why (2):
Batch verifier should use the execution pool to keep the number of goroutines performing heavy lifting in check. This approach is also followed in PaysetGroups for verifying the txns in a block.
The transaction handler cannot block until the transaction is verified, for that reason it also uses the exec pool in txHandler.
The handler and the verifier cannot both use the exec pool, this may lead to a deadlock. For this reason, the streaming approach is adopted here.

Other considerations and future work:

The signatures inside LogicSig (Msig or Sig) are not added to the main batch. Instead, they are batched in their own batch. This is a limitation and the performance can be improved by adding them to the top level batch. This requires the decoupling of the txn validity checks from the signature verification, and will be address in a different PR.

In the crypto_sign_ed25519_open_batch implementation, the validity of every transaction is checked in case the batch verification fails. This can be improved for all workflows:
a. in case we don't care with sig fails, and we reject the whole batch (Msig, block validation), we are doing the extra work of verifying each sig one by one when the batch fails.
b. in case we are which txn sig failed, in case of batch verification of the incoming txns, the verification can be done by checking logarithmic number of the sigs, instead we check linearly all of them.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions