Improve the efficiency of signature verification

[lurais]

Rationale

At present, there is a signature verification process in the transaction verification process. In order to improve the efficiency of signature verification, multiple transactions are currently verified in parallel. The number of CPU cores used in the parallel verification process is set to half of the current CPU core number. From the figure below, this setting is conservative and wastes node performance.

Here is the cpu usege stats and the transaction verification time cost. And it doesn't use too much CPU resource from the image while the average time of signature verification process may be more than 30ms.

System information
cpu：8 cores
memory: 16G
java-tron version: 4.7.2

And here is the cpu usage stats in the last two days, it used no more than 18% of the cpu resource.

Why should this feature exist?
To improve the efficiency of signature verification.

Implementation

Therefore, I propose to use all the cores in signature verification process to leverage the system performance and improve the efficiency of signature verification.

Are you willing to implement this feature?
yes

[tomatoishealthy]

It is best to list your hardware environment, especially the CPU.

In addition, how much percentage of performance is expected to be improved by this modification?

Is there any reference for adjusting the number of threads and CPU cores to 1:1?

[eodiandie]

I have 3 questions, as below:

1. will the CPU run at full capacity(100%) after you set the used number of cores in the signature verification process equal to the number of CPU cores?
2. if so. does this affect other jobs that are executed at the same time as the parallel verification of signatures?
3. Is there a strategy for measuring how many cores will be used in parallel verification of signatures that are best fit in the case?

[lurais]

cpu：8 cores
memory: 16G
java-tron version: 4.7.2

@guoquanwu yes, that has been add to the issue,100% percentage of signature verification performance is expected to be improved by this modification for the signature verification actually only depends on the cpu, so theoretically doubling the number of processing cores can reduce the processing time by one time. In addition, from the existing statistical charts listed above, the currently used cpu resources are low, it will not reach the bottleneck of the CPU even if the usage is doubled , so adjusting the number of CPU cores used will theoretically not lead to excessive resource consumption. Setting it to the same number as the number of CPU cores is because signature verification is a CPU-intensive operation that relies purely on the CPU.

[lurais]

1. capacity

@xq-lu 1. the CPU will not run at full capacity(100%) for current cpu is not running at 50% usage.
2.actually, as we can infer from the current cpu usage, the other jobs will not be affected.
3. theoretically, that is the number of cpu cores while it is CPU-intensive operation , but may be related to the specific tasks , if we want to find the best ,we can just pick a batch of transactions, and try to change the number of cores used to find it .

[halibobo1205]

1. The stats above, is catching up with the block or not ?
2. How many transactions waiting to be verified?
3. How long does a single verification take?

[halibobo1205]

Actually this parameter node.validateSignThreadNum is available, the default value is (CPUs + 1 )/2.

[lurais]

validateSignThreadNum

1.that is not catching up with the block.
2.about 2 transactions
3.less than 1ms, so it seems no need to set to the cpu cores as default, you can just set the validateSignThreadNum parameter when it takes much time @halibobo1205

[halibobo1205]

@lurais How much time will it save , can you describe it in more detail?

[halibobo1205]

@lurais How much time will it save , can you describe it in more detail?

@lurais Are there any other problems?

[lurais]

@lurais How much time will it save , can you describe it in more detail?

@halibobo1205 yes I have tested that and it saved about 10% of the time in block push operation with 8c 32g machine. And the time of verification signature reduced about 23%.
As it saved about 5% of the time in block push operation with 16c 32g machine.

[lurais]

@lurais How much time will it save , can you describe it in more detail?

@lurais Are there any other problems?

@halibobo1205 No other problems.