Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nsqd: benchmark various write/msg sizes #622

Merged
merged 1 commit into from
Aug 7, 2015

Conversation

Dieterbe
Copy link
Contributor

@Dieterbe Dieterbe commented Aug 7, 2015

because this has a big impact on performance

caveats:

  • if /tmp runs full, you'll get unrealistically high results,
    can be avoided by running with -benchtime=0.1s
  • for some reason on my system QueueGet > 1024 hangs
on my system:
BenchmarkDiskWrite16      300000               465 ns/op          34.36 MB/s
BenchmarkDiskWrite64      300000               476 ns/op         134.30 MB/s
BenchmarkDiskWrite256     300000               521 ns/op         491.03 MB/s
BenchmarkDiskWrite1024    200000               634 ns/op        1614.39 MB/s
BenchmarkDiskWrite4096    200000              1060 ns/op        3860.56 MB/s
BenchmarkDiskWrite16384    50000              2672 ns/op        6131.59 MB/s
BenchmarkDiskWrite65536    20000             10629 ns/op        6165.22 MB/s
BenchmarkDiskWrite262144            3000             39678 ns/op        6606.64 MB/s
BenchmarkDiskWrite1048576           1000            162506 ns/op        6452.50 MB/s
BenchmarkDiskWriteBuffered16    10000000                27.4 ns/op       583.05 MB/s
BenchmarkDiskWriteBuffered64     5000000                30.3 ns/op      2113.78 MB/s
BenchmarkDiskWriteBuffered256    1000000               108 ns/op        2367.44 MB/s
BenchmarkDiskWriteBuffered1024    500000               346 ns/op        2958.68 MB/s
BenchmarkDiskWriteBuffered4096    200000              1170 ns/op        3500.34 MB/s
BenchmarkDiskWriteBuffered16384    50000              2746 ns/op        5965.97 MB/s
BenchmarkDiskWriteBuffered65536    10000             10374 ns/op        6316.90 MB/s
BenchmarkDiskWriteBuffered262144            3000             41288 ns/op        6349.07 MB/s
BenchmarkDiskWriteBuffered1048576           1000            162323 ns/op        6459.78 MB/s

BenchmarkDiskQueueGet16   200000               956 ns/op          16.73 MB/s
BenchmarkDiskQueueGet64   200000              1021 ns/op          62.68 MB/s
BenchmarkDiskQueueGet256          100000              1252 ns/op         204.47 MB/s
BenchmarkDiskQueueGet1024         100000              2104 ns/op         486.49 MB/s

@mreiferson
Copy link
Member

Thanks! Looks like this needs a rebase

because this has a big impact on performance

caveats:
* if /tmp runs full, you'll get unrealistically high results,
can be avoided by running with -benchtime=0.1s
* for some reason on my system QueueGet > 1024 hangs

on my system:
BenchmarkDiskWrite16      300000               465 ns/op          34.36 MB/s
BenchmarkDiskWrite64      300000               476 ns/op         134.30 MB/s
BenchmarkDiskWrite256     300000               521 ns/op         491.03 MB/s
BenchmarkDiskWrite1024    200000               634 ns/op        1614.39 MB/s
BenchmarkDiskWrite4096    200000              1060 ns/op        3860.56 MB/s
BenchmarkDiskWrite16384    50000              2672 ns/op        6131.59 MB/s
BenchmarkDiskWrite65536    20000             10629 ns/op        6165.22 MB/s
BenchmarkDiskWrite262144            3000             39678 ns/op        6606.64 MB/s
BenchmarkDiskWrite1048576           1000            162506 ns/op        6452.50 MB/s
BenchmarkDiskWriteBuffered16    10000000                27.4 ns/op       583.05 MB/s
BenchmarkDiskWriteBuffered64     5000000                30.3 ns/op      2113.78 MB/s
BenchmarkDiskWriteBuffered256    1000000               108 ns/op        2367.44 MB/s
BenchmarkDiskWriteBuffered1024    500000               346 ns/op        2958.68 MB/s
BenchmarkDiskWriteBuffered4096    200000              1170 ns/op        3500.34 MB/s
BenchmarkDiskWriteBuffered16384    50000              2746 ns/op        5965.97 MB/s
BenchmarkDiskWriteBuffered65536    10000             10374 ns/op        6316.90 MB/s
BenchmarkDiskWriteBuffered262144            3000             41288 ns/op        6349.07 MB/s
BenchmarkDiskWriteBuffered1048576           1000            162323 ns/op        6459.78 MB/s

BenchmarkDiskQueueGet16   200000               956 ns/op          16.73 MB/s
BenchmarkDiskQueueGet64   200000              1021 ns/op          62.68 MB/s
BenchmarkDiskQueueGet256          100000              1252 ns/op         204.47 MB/s
BenchmarkDiskQueueGet1024         100000              2104 ns/op         486.49 MB/s
@Dieterbe
Copy link
Contributor Author

Dieterbe commented Aug 7, 2015

rebased.

mreiferson added a commit that referenced this pull request Aug 7, 2015
nsqd: benchmark various write/msg sizes
@mreiferson mreiferson merged commit 3406d83 into nsqio:master Aug 7, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants