More Benchmarks for your feedback

[jdvjdv82]

Hi, here is some feedback for you to get results on different architecture
Ubuntu 22.04
AMD Ryzen Threadripper 3960X
128 GB RAM

Simple

~/go/pkg/mod/github.com/alphadose/zenq@v0.0.0-20220510051127-b9702e399278$ go run benchmarks/simple/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 11.462µs
ZenQ Runner completed transfer in: 4.739µs
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 32.522µs
ZenQ Runner completed transfer in: 28.795µs
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 449.323µs
ZenQ Runner completed transfer in: 254.583µs
====================================================================

With Input Batch Size: 6000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 365.257669ms
ZenQ Runner completed transfer in: 447.14544ms
====================================================================

With Input Batch Size: 600000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 38.346896012s
ZenQ Runner completed transfer in: 48.59434176s
====================================================================

Selector

Chan Select Runner completed transfer in: 4m32.220766401s
ZenQ Select Runner did not complete transfer after 30min+ (tested both Ubuntu and Windows)

[alphadose]

Thanks, looking into it

[alphadose]

Also it would be great if you could also paste the output of go test -bench=. -benchmem benchmarks/simple/*.go

[jdvjdv82]

$ go test -bench=. -benchmem benchmarks/simple/*.go
goos: windows
goarch: amd64
cpu: AMD Ryzen Threadripper 3960X 24-Core Processor
Benchmark_Chan_NumWriters1_InputSize600-48                 31578             35531 ns/op               0 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters1_InputSize600-48                 32344             35370 ns/op               0 B/op          0 allocs/op
Benchmark_Chan_NumWriters3_InputSize60000-48                 138           8604078 ns/op              61 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters3_InputSize60000-48                 204           5953425 ns/op              18 B/op          0 allocs/op
Benchmark_Chan_NumWriters8_InputSize6000000-48                 1        1627559800 ns/op             288 B/op          3 allocs/op
Benchmark_ZenQ_NumWriters8_InputSize6000000-48                 4         310374800 ns/op            1248 B/op          3 allocs/op
Benchmark_Chan_NumWriters100_InputSize6000000-48               1        4324276500 ns/op           98752 B/op        263 allocs/op
Benchmark_ZenQ_NumWriters100_InputSize6000000-48               1        1333998600 ns/op           22880 B/op         55 allocs/op
Benchmark_Chan_NumWriters1000_InputSize7000000-48              1        5544090900 ns/op          543328 B/op       1910 allocs/op
Benchmark_ZenQ_NumWriters1000_InputSize7000000-48              1        1843502400 ns/op          114400 B/op        275 allocs/op
Benchmark_Chan_Million_Blocking_Writers-48                     1        10442282900 ns/op       553007936 B/op   1997923 allocs/op
Benchmark_ZenQ_Million_Blocking_Writers-48                     1        4816000900 ns/op        95865720 B/op     996070 allocs/op
PASS
ok      command-line-arguments  40.354s

[jdvjdv82]

• again 48 cpu in ms windows in GIT shell

$ go run benchmarks/simple/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 0s
ZenQ Runner completed transfer in: 0s
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 0s
ZenQ Runner completed transfer in: 0s
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 499.9µs
ZenQ Runner completed transfer in: 499.3µs
====================================================================

With Input Batch Size: 6000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 364.5002ms
ZenQ Runner completed transfer in: 350.999ms
====================================================================

With Input Batch Size: 600000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 38.4405129s
ZenQ Runner completed transfer in: 29.0519996s
====================================================================

second test

did not finish

[alphadose]

Got it, the current Select implementation is based on polling hence that is slow as evident in the second test not finishing

also interesting to note for high core amd64 CPU , SPSC mode ZenQ is slower than channels for large input sizes (6 * 10^8), this shall be improved too

Working on the improvements, will be released in next patch

[jdvjdv82]

yep, just let me know here. I will test on both platforms and different PCs again. I did not finish also on laptop 8 core i7.

[alphadose]

@jdvjdv82 Hello, I have updated the codebase and would like your help in benchmarking across both windows and ubuntu if possible
Pull the latest main branch
I need the outputs of the following commands

$ go run benchmarks/selector/main.go
$ go run benchmarks/simple/main.go
$ go test -bench=. -benchmem benchmarks/simple/*.go

Finally, I also need your hardware specs, that would really help me

Thanks in advance :)

[jdvjdv82]

Hello @alphadose,
cpu: AMD Ryzen Threadripper 3960X 24-Core Processor
128 GB RAM, many NVME pcie 4.0 SSDs
following test is on ms windows 11

$ go run benchmarks/selector/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 0s
ZenQ Select Runner completed transfer in: 0s
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 501.5µs
ZenQ Select Runner completed transfer in: 1.0005ms
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 1.4982ms
ZenQ Select Runner completed transfer in: 10.0003ms
====================================================================

With Input Batch Size: 600000 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 264.5006ms
ZenQ Select Runner completed transfer in: 34.0929588s
====================================================================

$ go run benchmarks/simple/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 0s
ZenQ Runner completed transfer in: 0s
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 0s
ZenQ Runner completed transfer in: 0s
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 500.4µs
ZenQ Runner completed transfer in: 501.2µs
====================================================================

With Input Batch Size: 6000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 423.4996ms
ZenQ Runner completed transfer in: 356.4958ms
====================================================================

With Input Batch Size: 600000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 32.5962971s
ZenQ Runner completed transfer in: 19.3549989s
====================================================================

$ go test -bench=. -benchmem benchmarks/simple/*.go
goos: windows
goarch: amd64
cpu: AMD Ryzen Threadripper 3960X 24-Core Processor 
Benchmark_Chan_NumWriters1_InputSize600-48                 34432             34547 ns/op               0 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters1_InputSize600-48                 38709             33545 ns/op               0 B/op          0 allocs/op
Benchmark_Chan_NumWriters3_InputSize60000-48                 123           9621558 ns/op              64 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters3_InputSize60000-48                 198           5724741 ns/op              67 B/op          0 allocs/op
Benchmark_Chan_NumWriters8_InputSize6000000-48                 1        1634815600 ns/op             704 B/op          4 allocs/op
Benchmark_ZenQ_NumWriters8_InputSize6000000-48                 4         291124900 ns/op            2646 B/op          6 allocs/op
Benchmark_Chan_NumWriters100_InputSize6000000-48               1        3884906000 ns/op          103104 B/op        271 allocs/op
Benchmark_ZenQ_NumWriters100_InputSize6000000-48               3         458999667 ns/op           43506 B/op        103 allocs/op
Benchmark_Chan_NumWriters1000_InputSize7000000-48              1        4842098300 ns/op          493344 B/op       1835 allocs/op
Benchmark_ZenQ_NumWriters1000_InputSize7000000-48              3         514832700 ns/op           77250 B/op        107 allocs/op
Benchmark_Chan_Million_Blocking_Writers-48                     1        9477141800 ns/op        552752480 B/op   1997281 allocs/op
Benchmark_ZenQ_Million_Blocking_Writers-48                     1        9756999400 ns/op        56204904 B/op    1000621 allocs/op
PASS
ok      command-line-arguments  45.755s

[alphadose]

Interesting that ZenQ select is really slow on windows for large batch sizes even with 24 cores, point noted

[jdvjdv82]

ubuntu

go run benchmarks/selector/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 43.181µs
ZenQ Select Runner completed transfer in: 180.5µs
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 225.264µs
ZenQ Select Runner completed transfer in: 1.178731ms
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 1.478737ms
ZenQ Select Runner completed transfer in: 9.839155ms
====================================================================

With Input Batch Size: 600000 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 238.86788ms
ZenQ Select Runner completed transfer in: 1.529784701s
====================================================================

go run benchmarks/simple/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 8.456µs
ZenQ Runner completed transfer in: 3.547µs
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 32.832µs
ZenQ Runner completed transfer in: 25.909µs
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 426.674µs
ZenQ Runner completed transfer in: 256.042µs
====================================================================

With Input Batch Size: 6000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 401.525353ms
ZenQ Runner completed transfer in: 489.642878ms
====================================================================

With Input Batch Size: 600000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 39.195709943s
ZenQ Runner completed transfer in: 46.277523423s
====================================================================

$ go test -bench=. -benchmem benchmarks/simple/*.go
goos: linux
goarch: amd64
cpu: AMD Ryzen Threadripper 3960X 24-Core Processor 
Benchmark_Chan_NumWriters1_InputSize600-48                 39223             28165 ns/op               0 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters1_InputSize600-48                 38683             26149 ns/op               0 B/op          0 allocs/op
Benchmark_Chan_NumWriters3_InputSize60000-48                 100          13115482 ns/op             235 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters3_InputSize60000-48                 140           7945363 ns/op             133 B/op          0 allocs/op
Benchmark_Chan_NumWriters8_InputSize6000000-48                 1        2155760648 ns/op             288 B/op          3 allocs/op
Benchmark_ZenQ_NumWriters8_InputSize6000000-48                 3         527535951 ns/op             832 B/op          2 allocs/op
Benchmark_Chan_NumWriters100_InputSize6000000-48               1        4570674913 ns/op           90976 B/op        247 allocs/op
Benchmark_ZenQ_NumWriters100_InputSize6000000-48               1        1203629449 ns/op           52824 B/op        219 allocs/op
Benchmark_Chan_NumWriters1000_InputSize7000000-48              1        7451312608 ns/op          507872 B/op       1820 allocs/op
Benchmark_ZenQ_NumWriters1000_InputSize7000000-48              1        1880767207 ns/op          512120 B/op       1923 allocs/op
Benchmark_Chan_Million_Blocking_Writers-48                     1        11394638647 ns/op       552651808 B/op   1996629 allocs/op
Benchmark_ZenQ_Million_Blocking_Writers-48                     1        73696722622 ns/op       46606496 B/op    1000246 allocs/op
PASS
ok      command-line-arguments  111.863s

[jdvjdv82]

go job!!! memory allocation is much better and that matters a lot

[alphadose]

also zenq performs worse than channels for large size SPSC in ubuntu, another interesting thing :3

[alphadose]

@jdvjdv82 I have made some considerable improvements, it should now perform better in both windows and ubuntu

Pls help me out with the benchmarking once more :)

$ go test -bench=. -benchmem benchmarks/simple/*.go

[jdvjdv82]

hi, just think about how you doing ;-)

$ go test -bench=. -benchmem benchmarks/simple/*.go
goos: windows
goarch: amd64
cpu: AMD Ryzen Threadripper 3960X 24-Core Processor 
Benchmark_Chan_NumWriters1_InputSize600-48                 35980             33519 ns/op               0 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters1_InputSize600-48                 28880             41084 ns/op               0 B/op          0 allocs/op
Benchmark_Chan_NumWriters3_InputSize60000-48                 142           8285233 ns/op              52 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters3_InputSize60000-48                 226           5455738 ns/op              22 B/op          0 allocs/op
Benchmark_Chan_NumWriters8_InputSize6000000-48                 1        1528500500 ns/op             192 B/op          2 allocs/op
Benchmark_ZenQ_NumWriters8_InputSize6000000-48                 4         262374500 ns/op             416 B/op          1 allocs/op
Benchmark_Chan_NumWriters100_InputSize6000000-48               1        4317605800 ns/op           96512 B/op        263 allocs/op
Benchmark_ZenQ_NumWriters100_InputSize6000000-48               4         254250575 ns/op           40348 B/op         92 allocs/op
Benchmark_Chan_NumWriters1000_InputSize7000000-48              1        5933000000 ns/op          517856 B/op       1887 allocs/op
Benchmark_ZenQ_NumWriters1000_InputSize7000000-48              2         511749150 ns/op          240336 B/op        297 allocs/op
Benchmark_Chan_Million_Blocking_Writers-48                     1        10942000900 ns/op       552677888 B/op   1996974 allocs/op
Benchmark_ZenQ_Million_Blocking_Writers-48                     1        2281501700 ns/op        125432480 B/op    999394 allocs/op
PASS
ok      command-line-arguments  36.985s

With Input Batch Size: 600 and Num Concurrent Writers: 1
With Input Batch Size: 60 and Num Concurrent Writers: 1
Native Channel Runner completed transfer in: 0s
ZenQ Runner completed transfer in: 0sfer in: 497.2µs
====================================================================
====================================================================
With Input Batch Size: 6000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 498.8µs
ZenQ Runner completed transfer in: 0s
====================================================================

With Input Batch Size: 6000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 382.9995ms
ZenQ Runner completed transfer in: 339.4975ms
====================================================================

With Input Batch Size: 600000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 36.4965056s
ZenQ Runner completed transfer in: 28.2475032s
====================================================================

$ go run benchmarks/selector/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 0s
ZenQ Select Runner completed transfer in: 498.7µs
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 501.8µs
ZenQ Select Runner completed transfer in: 996.8µs
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 1.5028ms
ZenQ Select Runner completed transfer in: 6.9995ms
====================================================================

With Input Batch Size: 600000 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 240.5839ms
ZenQ Select Runner completed transfer in: 9.3453999s
====================================================================

[jdvjdv82]

ubuntu

go test -bench=. -benchmem benchmarks/simple/*.go
goos: linux
goarch: amd64
cpu: AMD Ryzen Threadripper 3960X 24-Core Processor 
Benchmark_Chan_NumWriters1_InputSize600-48                 29367             63828 ns/op               0 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters1_InputSize600-48                 26434             42871 ns/op               0 B/op          0 allocs/op
Benchmark_Chan_NumWriters3_InputSize60000-48                 100          14247316 ns/op             221 B/op          0 allocs/op
Benchmark_ZenQ_NumWriters3_InputSize60000-48                 165           6944204 ns/op              55 B/op          0 allocs/op
Benchmark_Chan_NumWriters8_InputSize6000000-48                 1        2284501030 ns/op             192 B/op          2 allocs/op
Benchmark_ZenQ_NumWriters8_InputSize6000000-48                 3         466321840 ns/op            3704 B/op          7 allocs/op
Benchmark_Chan_NumWriters100_InputSize6000000-48               1        4229892337 ns/op           85056 B/op        234 allocs/op
Benchmark_ZenQ_NumWriters100_InputSize6000000-48               2         555276258 ns/op           92532 B/op        213 allocs/op
Benchmark_Chan_NumWriters1000_InputSize7000000-48              1        7424658850 ns/op          519648 B/op       1859 allocs/op
Benchmark_ZenQ_NumWriters1000_InputSize7000000-48              2         735947482 ns/op          511892 B/op        557 allocs/op
Benchmark_Chan_Million_Blocking_Writers-48                     1        11491585357 ns/op       552311424 B/op   1995873 allocs/op
Benchmark_ZenQ_Million_Blocking_Writers-48                     1        2336399737 ns/op        126273784 B/op   1000632 allocs/op
PASS

go run benchmarks/simple/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 8.046µs
ZenQ Runner completed transfer in: 4.65µs
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 35.162µs
ZenQ Runner completed transfer in: 28.218µs
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 450.423µs
ZenQ Runner completed transfer in: 252.888µs
====================================================================

With Input Batch Size: 6000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 350.160551ms
ZenQ Runner completed transfer in: 392.064067ms
====================================================================

With Input Batch Size: 600000000 and Num Concurrent Writers: 1

Native Channel Runner completed transfer in: 38.751119088s
ZenQ Runner completed transfer in: 57.735321409s
====================================================================

go run benchmarks/selector/main.go
With Input Batch Size: 60 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 53.071µs
ZenQ Select Runner completed transfer in: 169.062µs
====================================================================

With Input Batch Size: 600 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 247.722µs
ZenQ Select Runner completed transfer in: 1.004665ms
====================================================================

With Input Batch Size: 6000 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 1.787749ms
ZenQ Select Runner completed transfer in: 9.425709ms
====================================================================

With Input Batch Size: 600000 and Num Concurrent Writers: 4

Chan Select Runner completed transfer in: 284.181804ms
ZenQ Select Runner completed transfer in: 1.733328132s
====================================================================