Skip to content

F# micro benchmarks #4051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 18, 2024
Merged

F# micro benchmarks #4051

merged 3 commits into from
Mar 18, 2024

Conversation

psfinaki
Copy link
Member

@psfinaki psfinaki commented Mar 12, 2024

This is the first set of F# micro benchmarks.

I am not sure how to best locate it, happy to rearrange things if needed. F# benchmarks cannot be in the MicroBenchmarks.csproj solution because .cs and .fs files don't mix, but otherwise the benchmarks are quite similar to C# micro benchmarks.

Here are the local results for reference:

// * Summary *

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22621.3155/22H2/2022Update/SunValley2)
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK 9.0.100-preview.3.24161.4
  [Host]     : .NET 9.0.0 (9.0.24.16003), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI DEBUG
  Job-GUWNBV : .NET 9.0.0 (9.0.24.16003), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

PowerPlanMode=00000000-0000-0000-0000-000000000000  IterationTime=250ms  MaxIterationCount=20
MinIterationCount=15  WarmupCount=1
Method Length Mean Error StdDev Median Min Max Gen0 Gen1 Allocated
ListRemoveAtBeginning 1000 1.616 ns 0.1624 ns 0.1738 ns 1.583 ns 1.327 ns 1.992 ns - - -
ListRemoveAtEnd 1000 3,868.985 ns 74.5617 ns 69.7450 ns 3,857.379 ns 3,783.758 ns 3,991.502 ns 5.0943 0.5503 31968 B
ListRemoveManyAtBeginning 1000 6.548 ns 0.5164 ns 0.5947 ns 6.539 ns 5.700 ns 7.801 ns - - -
ListRemoveManyAtEnd 1000 4,050.353 ns 176.3517 ns 188.6943 ns 3,989.879 ns 3,786.722 ns 4,549.706 ns 5.0351 0.5360 31648 B
ListInsertAtBeginning 1000 5.124 ns 0.1237 ns 0.1215 ns 5.123 ns 4.956 ns 5.371 ns 0.0051 - 32 B
ListInsertAtEnd 1000 3,779.738 ns 120.7858 ns 129.2395 ns 3,768.250 ns 3,569.024 ns 4,089.195 ns 5.0944 0.5410 32000 B
ListInsertManyAtBeginning 1000 110.121 ns 6.4632 ns 7.1838 ns 107.314 ns 101.631 ns 126.613 ns 0.1146 - 720 B
ListInsertManyAtEnd 1000 3,936.979 ns 96.5846 ns 103.3444 ns 3,912.720 ns 3,813.956 ns 4,180.280 ns 5.1475 0.5428 32368 B
ListUpdateAtBeginning 1000 5.610 ns 0.4850 ns 0.4981 ns 5.527 ns 5.021 ns 7.000 ns 0.0051 - 32 B
ListUpdateEnd 1000 3,921.972 ns 75.1998 ns 70.3419 ns 3,930.851 ns 3,819.301 ns 4,042.876 ns 5.0998 0.5402 32000 B
ArrayRemoveAtBeginning 1000 203.423 ns 22.9027 ns 24.5056 ns 190.984 ns 181.279 ns 256.535 ns 0.6412 - 4024 B
ArrayRemoveAtEnd 1000 207.105 ns 29.4275 ns 32.7086 ns 189.636 ns 179.492 ns 275.996 ns 0.6414 - 4024 B
ArrayRemoveManyAtBeginning 1000 201.721 ns 22.7179 ns 25.2509 ns 192.449 ns 175.102 ns 252.983 ns 0.6351 - 3984 B
ArrayRemoveManyAtEnd 1000 234.299 ns 25.7745 ns 28.6482 ns 230.533 ns 193.544 ns 280.782 ns 0.6347 - 3984 B
ArrayInsertAtBeginning 1000 267.162 ns 65.4301 ns 75.3494 ns 251.573 ns 181.631 ns 441.626 ns 0.6423 - 4032 B
ArrayInsertAtEnd 1000 189.282 ns 7.0473 ns 7.5405 ns 186.810 ns 181.959 ns 206.411 ns 0.6419 - 4032 B
ArrayInsertManyAtBeginning 1000 279.181 ns 18.9594 ns 21.0734 ns 272.518 ns 254.818 ns 321.856 ns 0.7212 - 4528 B
ArrayInsertManyAtEnd 1000 276.366 ns 10.4415 ns 10.7227 ns 278.410 ns 259.599 ns 296.223 ns 0.7214 - 4528 B
ArrayUpdateAtBeginning 1000 187.333 ns 5.0145 ns 5.3654 ns 187.151 ns 180.855 ns 195.878 ns 0.6411 - 4024 B
ArrayUpdateEnd 1000 179.605 ns 2.4224 ns 2.0228 ns 179.593 ns 176.956 ns 182.829 ns 0.6413 - 4024 B
SeqBaseline 1000 3,216.800 ns 42.5950 ns 39.8433 ns 3,223.691 ns 3,150.432 ns 3,296.924 ns 5.0925 0.2906 32000 B
SeqRemoveAtBeginning 1000 10,337.594 ns 301.2265 ns 322.3090 ns 10,281.040 ns 9,903.277 ns 10,993.830 ns 5.0953 0.5342 32176 B
SeqRemoveAtEnd 1000 10,891.685 ns 473.0141 ns 544.7238 ns 10,815.612 ns 9,937.074 ns 11,964.077 ns 5.0949 0.5271 32176 B
SeqRemoveManyAtBeginning 1000 10,671.222 ns 394.3573 ns 404.9759 ns 10,595.591 ns 10,096.444 ns 11,741.624 ns 5.0735 0.5248 31888 B
SeqRemoveManyAtEnd 1000 6,990.076 ns 365.6366 ns 406.4041 ns 6,969.629 ns 6,352.761 ns 7,735.200 ns 0.0265 - 208 B
SeqInsertAtBeginning 1000 7,916.498 ns 490.9468 ns 504.1663 ns 7,771.623 ns 7,370.784 ns 9,149.015 ns 0.0286 - 224 B
SeqInsertAtEnd 1000 8,009.211 ns 243.9363 ns 261.0091 ns 8,020.303 ns 7,540.850 ns 8,440.083 ns 0.0349 - 224 B
SeqInsertManyAtBeginning 1000 8,171.950 ns 276.8284 ns 284.2824 ns 8,062.950 ns 7,885.103 ns 8,950.531 ns 0.0970 - 696 B
SeqInsertManyAtEnd 1000 7,942.260 ns 95.9819 ns 85.0855 ns 7,922.826 ns 7,818.007 ns 8,136.157 ns 0.0937 - 696 B
SeqUpdateAtBeginning 1000 8,161.467 ns 672.9911 ns 748.0279 ns 7,880.177 ns 7,339.991 ns 9,786.487 ns 0.0289 - 208 B
SeqUpdateEnd 1000 7,407.959 ns 123.8897 ns 109.8250 ns 7,399.635 ns 7,177.457 ns 7,603.276 ns 0.0308 - 208 B
ListRemoveAtBeginning 10000 1.729 ns 0.0271 ns 0.0253 ns 1.730 ns 1.698 ns 1.778 ns - - -
ListRemoveAtEnd 10000 55,994.958 ns 3,146.2913 ns 3,231.0095 ns 55,435.090 ns 51,096.534 ns 64,542.507 ns 50.8363 19.8063 319968 B
ListRemoveManyAtBeginning 10000 8.621 ns 1.6799 ns 1.9346 ns 8.595 ns 5.904 ns 11.651 ns - - -
ListRemoveManyAtEnd 10000 57,441.159 ns 3,746.0225 ns 4,008.2021 ns 57,107.127 ns 52,653.959 ns 67,699.604 ns 50.9050 18.9480 319648 B
ListInsertAtBeginning 10000 5.170 ns 0.1980 ns 0.2119 ns 5.220 ns 4.766 ns 5.542 ns 0.0051 - 32 B
ListInsertAtEnd 10000 54,027.115 ns 4,017.0802 ns 4,125.2456 ns 52,520.665 ns 50,131.303 ns 65,083.777 ns 50.7979 19.1489 320000 B
ListInsertManyAtBeginning 10000 104.324 ns 2.2809 ns 2.5352 ns 103.801 ns 101.434 ns 110.111 ns 0.1146 - 720 B
ListInsertManyAtEnd 10000 52,971.383 ns 423.8507 ns 396.4702 ns 52,920.079 ns 52,371.179 ns 53,701.329 ns 50.8721 17.8571 320368 B
ListUpdateAtBeginning 10000 6.066 ns 0.7600 ns 0.8753 ns 6.071 ns 5.117 ns 7.938 ns 0.0051 - 32 B
ListUpdateEnd 10000 60,177.101 ns 7,051.4143 ns 8,120.4192 ns 55,872.192 ns 52,225.962 ns 74,563.213 ns 50.9511 19.2482 320000 B
ArrayRemoveAtBeginning 10000 2,218.811 ns 123.5454 ns 137.3203 ns 2,182.505 ns 2,047.877 ns 2,482.350 ns 6.3269 - 40024 B
ArrayRemoveAtEnd 10000 2,307.523 ns 165.5330 ns 190.6280 ns 2,237.361 ns 2,054.620 ns 2,640.811 ns 6.3288 - 40024 B
ArrayRemoveManyAtBeginning 10000 2,249.009 ns 124.6084 ns 127.9636 ns 2,234.685 ns 2,065.604 ns 2,497.313 ns 6.3272 - 39984 B
ArrayRemoveManyAtEnd 10000 3,959.457 ns 335.1096 ns 385.9127 ns 3,979.007 ns 3,355.837 ns 4,569.259 ns 6.3272 - 39984 B
ArrayInsertAtBeginning 10000 3,014.439 ns 83.3425 ns 81.8535 ns 2,991.808 ns 2,917.867 ns 3,221.126 ns 6.3221 - 40032 B
ArrayInsertAtEnd 10000 2,359.411 ns 122.9114 ns 141.5449 ns 2,338.829 ns 2,132.416 ns 2,633.225 ns 6.3271 - 40032 B
ArrayInsertManyAtBeginning 10000 2,388.609 ns 182.0050 ns 186.9058 ns 2,319.494 ns 2,222.466 ns 2,926.148 ns 6.4048 - 40528 B
ArrayInsertManyAtEnd 10000 2,538.255 ns 239.2046 ns 265.8753 ns 2,388.729 ns 2,247.393 ns 3,200.300 ns 6.4064 - 40528 B
ArrayUpdateAtBeginning 10000 2,354.519 ns 164.8145 ns 169.2523 ns 2,330.114 ns 2,109.747 ns 2,818.425 ns 6.3259 - 40024 B
ArrayUpdateEnd 10000 2,153.689 ns 34.7975 ns 34.1758 ns 2,159.791 ns 2,071.296 ns 2,198.154 ns 6.3234 - 40024 B
SeqBaseline 10000 44,720.990 ns 983.3879 ns 1,132.4710 ns 44,350.724 ns 43,210.505 ns 46,758.104 ns 50.8475 16.9492 320000 B
SeqRemoveAtBeginning 10000 168,129.818 ns 22,150.0666 ns 25,508.0497 ns 166,207.135 ns 131,815.000 ns 226,747.708 ns 50.5208 18.7500 320176 B
SeqRemoveAtEnd 10000 123,462.309 ns 3,965.2661 ns 4,242.7903 ns 123,021.256 ns 116,355.068 ns 133,222.241 ns 50.6757 18.5811 320176 B
SeqRemoveManyAtBeginning 10000 159,074.707 ns 26,664.8330 ns 30,707.2613 ns 152,280.261 ns 122,937.819 ns 225,845.344 ns 50.3827 19.1327 319888 B
SeqRemoveManyAtEnd 10000 87,332.901 ns 11,977.1455 ns 13,792.8986 ns 81,391.290 ns 74,691.589 ns 118,064.761 ns - - 208 B
SeqInsertAtBeginning 10000 75,088.242 ns 1,863.0427 ns 1,993.4348 ns 74,794.924 ns 70,845.657 ns 79,796.567 ns - - 224 B
SeqInsertAtEnd 10000 81,058.068 ns 3,672.3834 ns 4,229.1222 ns 80,052.443 ns 76,607.528 ns 91,412.557 ns - - 224 B
SeqInsertManyAtBeginning 10000 91,625.249 ns 4,629.0714 ns 5,145.2006 ns 91,281.250 ns 83,250.549 ns 101,067.926 ns - - 696 B
SeqInsertManyAtEnd 10000 94,282.410 ns 12,086.8937 ns 13,919.2847 ns 87,260.445 ns 76,516.477 ns 123,156.350 ns - - 696 B
SeqUpdateAtBeginning 10000 88,261.553 ns 8,077.3063 ns 9,301.8380 ns 89,425.156 ns 73,779.094 ns 106,600.625 ns - - 208 B

@psfinaki psfinaki marked this pull request as ready for review March 13, 2024 12:00
@psfinaki
Copy link
Member Author

/azp run

Copy link

Commenter does not have sufficient privileges for PR 4051 in repo dotnet/performance

@psfinaki psfinaki closed this Mar 13, 2024
@psfinaki psfinaki reopened this Mar 13, 2024
@LoopedBard3 LoopedBard3 self-assigned this Mar 13, 2024
@LoopedBard3
Copy link
Member

Generally looks good for a starting point to me. Going to check with the team tomorrow to determine if this is the right place for it to live or if a different folder would make more sense.

@LoopedBard3
Copy link
Member

Hello, got some feedback on the location. We think that the work should be put in its own folder under src/benchmarks/micro-fsharp as it doesn't really fit in the real-world scenarios but the current micro folder is definitely more focused on the C# microbenchmarks. If you can move it to the src/benchmarks/micro-fsharp location, along with creating a MicrobenchmarkFSharp solution (.sln), that would be great. Let us know if you have any other questions or comments on this 👍.

@psfinaki
Copy link
Member Author

Hey, thanks for the feedback - no problem, I will reorganize the files as suggested, stay tuned :)

@psfinaki psfinaki closed this Mar 18, 2024
@psfinaki psfinaki reopened this Mar 18, 2024
@psfinaki psfinaki closed this Mar 18, 2024
@psfinaki psfinaki reopened this Mar 18, 2024
@psfinaki
Copy link
Member Author

Hey @LoopedBard3, I updated the benchmarks as asked.

The CI is failing now but for something else - I cannot understand if it's related or not, but doesn't seem to be flaky - I reran it a few times.

@LoopedBard3
Copy link
Member

These failures are unrelated, so you are all good 👍. Are you ready for this to be merged in?

@psfinaki
Copy link
Member Author

Yeah, all good from my side :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants