Skip to content
This repository has been archived by the owner on Jun 6, 2023. It is now read-only.

Scenario tests measuring pre-commit gas cost, by batch size. #1426

Merged
merged 3 commits into from
May 27, 2021

Conversation

anorth
Copy link
Member

@anorth anorth commented May 20, 2021

Adds a scenario test that measures the gas cost of PreCommitSectorBatch for various batch sizes. When pre-committing 1000 sectors, at the maximum batch size, the per-sector cost is reduced to 15% vs batch size of 1 (so 6.5x improvement). With smaller sector counts the relative improvement of large batches is even better (because the HAMT stays smaller).

Sector count	1,000
Batch size	IPLD cost	Calls cost	Total per sector	Total per batch
1	9,670,098,646	29,233,000	9,699,331	9,699,331
2	5,754,837,950	14,616,500	5,769,454	11,538,908
4	3,735,873,808	7,308,250	3,743,182	14,972,728
8	2,672,493,076	3,654,125	2,676,147	21,409,176
16	2,002,461,804	1,841,679	2,004,303	32,068,848
32	1,487,183,156	935,456	1,488,118	47,619,776

Note that this underestimates the improvement to the current mainnet code, because the single-sector case enjoyed some state access optimisations during implementation in terms of the new PreCommitSectorBatch.

The earlier commits in this PR implement some clean-up and I was getting my head around the VM stats. Feel free to ask for reversion of or push a commit reverting anything that doesn't work for you. The TestMeasurePoRepGas output now looks like

--------------------- Batch 0 ---------------------
fil/5/storageminer:7: calls: 200  gets: 2,984  puts: 1,385  read: 841,713  written: 840,025  avg gets: 14.92, avg puts: 6.93
fil/5/storageminer:7: ipld gas: 1,432,991,578 call gas: 5,846,600
  fil/5/storagemarket:8: calls: 200  gets: 600  puts: 200  read: 168,091  written: 168,600  avg gets: 3.00, avg puts: 1.00
  fil/5/storagemarket:8: ipld gas: 281,139,200 call gas: 5,846,600
  fil/5/storagepower:8: calls: 200  gets: 1,369  puts: 985  read: 501,814  written: 502,825  avg gets: 6.84, avg puts: 4.93
  fil/5/storagepower:8: ipld gas: 839,487,748 call gas: 5,846,600
fil/5/cron:2: calls: 1  gets: 182  puts: 80  read: 181,650  written: 83,922  avg gets: 182.00, avg puts: 80.00
fil/5/cron:2: ipld gas: 129,518,244 call gas: 29,233
  fil/5/storagepower:5: calls: 1  gets: 172  puts: 71  read: 181,220  written: 81,930  avg gets: 172.00, avg puts: 71.00
  fil/5/storagepower:5: ipld gas: 125,419,594 call gas: 29,233
    fil/5/storageminer:17: calls: 1  gets: 148  puts: 64  read: 153,041  written: 79,579  avg gets: 148.00, avg puts: 64.00
    fil/5/storageminer:17: ipld gas: 119,968,996 call gas: 29,233
      fil/5/reward:3: calls: 1  gets: 2  puts: 1  read: 175  written: 843  avg gets: 2.00, avg puts: 1.00
      fil/5/reward:3: ipld gas: 1,330,454 call gas: 29,233
      fil/5/storagepower:9: calls: 1  gets: 2  puts: 1  read: 208  written: 843  avg gets: 2.00, avg puts: 1.00
      fil/5/storagepower:9: ipld gas: 1,330,454 call gas: 29,233
      fil/5/storagepower:6: calls: 1  gets: 3  puts: 2  read: 221  written: 1,005  avg gets: 3.00, avg puts: 2.00
      fil/5/storagepower:6: ipld gas: 1,700,366 call gas: 29,233
    fil/5/reward:4: calls: 1  gets: 2  puts: 2  read: 175  written: 962  avg gets: 2.00, avg puts: 2.00
    fil/5/reward:4: ipld gas: 1,569,224 call gas: 29,233
  fil/5/storagemarket:9: calls: 1  gets: 8  puts: 8  read: 362  written: 1,149  avg gets: 8.00, avg puts: 8.00
  fil/5/storagemarket:9: ipld gas: 2,768,196 call gas: 29,233

@anorth anorth requested a review from ZenGround0 May 20, 2021 10:14
@anorth anorth force-pushed the anorth/batchtests branch from 08fadca to 0533105 Compare May 20, 2021 10:14
@codecov-commenter
Copy link

codecov-commenter commented May 20, 2021

Codecov Report

Merging #1426 (482e282) into master (d03ede5) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #1426   +/-   ##
======================================
  Coverage    70.2%   70.2%           
======================================
  Files          72      72           
  Lines        7779    7779           
======================================
  Hits         5461    5461           
  Misses       1435    1435           
  Partials      883     883           

@anorth anorth force-pushed the anorth/batchtests branch 2 times, most recently from b81e13d to 9783ce5 Compare May 27, 2021 03:20
@ZenGround0 ZenGround0 merged commit 6a8c4e6 into master May 27, 2021
@anorth anorth deleted the anorth/batchtests branch January 11, 2022 04:44
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants