Skip to content
This repository was archived by the owner on Jun 20, 2025. It is now read-only.

Improved sync/async benchmarks #48

Merged
merged 5 commits into from
Jul 18, 2018
Merged

Conversation

segabriel
Copy link
Member

No description provided.

@segabriel segabriel requested review from artem-v and snripa July 17, 2018 19:54
@segabriel segabriel added the ready for review ready for review label Jul 17, 2018
@segabriel segabriel added work in progress work in progress and removed ready for review ready for review labels Jul 18, 2018

latch.await(settings.executionTaskDuration().toMillis(), TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better just propagate with Exceptions.propagate(e) . Can u put right after this comment an improvement whihch this parallel thing is giving to us. I heard u told it's like 2x perf boost. Ok, can u put the stats right here. Like, here it was, here it became.

@@ -279,7 +293,7 @@ public final void runForAsync(Function<SELF, Function<Long, Publisher<?>>> func)

BiFunction<Long, T, Publisher<?>> unitOfWork = func.apply(self);

Flux.interval(settings.rampUpInterval())
Flux.interval(Duration.ZERO, settings.rampUpInterval())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@@ -239,12 +247,18 @@ public final void runForAsync(Function<SELF, Function<Long, Publisher<?>>> func)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you have said runForAsync doesn't work well with this parallel() runOn() trick . Plz revert .

@segabriel
Copy link
Member Author

I run SyncExampleBenchmarksRunner on my laptop, and before these changes, I received the following results:

i.s.b.e.SyncExampleBenchmarksRunner-timer
             count = 57722532
         mean rate = 959152.09 calls/second
     1-minute rate = 925116.24 calls/second
     5-minute rate = 881552.39 calls/second
    15-minute rate = 870389.79 calls/second
               min = 649.00 nanoseconds
               max = 4780.00 nanoseconds
              mean = 684.39 nanoseconds
            stddev = 158.46 nanoseconds
            median = 660.00 nanoseconds
              75% <= 668.00 nanoseconds
              95% <= 816.00 nanoseconds
              98% <= 899.00 nanoseconds
              99% <= 941.00 nanoseconds
            99.9% <= 3837.00 nanoseconds

And after:

-- Timers ----------------------------------------------------------------------
i.s.b.e.SyncExampleBenchmarksRunner-timer
             count = 122485880
         mean rate = 2035577.46 calls/second
     1-minute rate = 1940461.86 calls/second
     5-minute rate = 1809094.25 calls/second
    15-minute rate = 1776252.26 calls/second
               min = 657.00 nanoseconds
               max = 8453.00 nanoseconds
              mean = 1000.55 nanoseconds
            stddev = 293.65 nanoseconds
            median = 993.00 nanoseconds
              75% <= 1020.00 nanoseconds
              95% <= 1048.00 nanoseconds
              98% <= 1061.00 nanoseconds
              99% <= 1076.00 nanoseconds
            99.9% <= 8453.00 nanoseconds

@segabriel segabriel added ready for review ready for review and removed work in progress work in progress labels Jul 18, 2018
@artem-v
Copy link
Contributor

artem-v commented Jul 18, 2018

I run SyncExampleBenchmarksRunner on my laptop, and before these changes, I received the following results:

Have you noticed one thing: tps rate became larger 2x times, but latency is worsen. That's strange. I run sync exmple runner locally and indeed tps went top 2x times , but latency remained the same, just saying you stats are confusing. But we will merge this PR because it's really improves sync tests. Good job, thanks.

PS on my laptop
Before:

i.s.b.e.SyncExampleBenchmarksRunner-timer
             count = 12287647
         mean rate = 1024352.53 calls/second
     1-minute rate = 1011890.98 calls/second
     5-minute rate = 1009587.47 calls/second
    15-minute rate = 1009188.40 calls/second
               min = 581.00 nanoseconds
               max = 3283.00 nanoseconds
              mean = 635.63 nanoseconds
            stddev = 138.45 nanoseconds
            median = 603.00 nanoseconds
              75% <= 608.00 nanoseconds
              95% <= 833.00 nanoseconds
              98% <= 1080.00 nanoseconds
              99% <= 1261.00 nanoseconds
            99.9% <= 3283.00 nanoseconds

After:

i.s.b.e.SyncExampleBenchmarksRunner-timer
             count = 154712503
         mean rate = 2881602.55 calls/second
     1-minute rate = 2826544.70 calls/second
     5-minute rate = 2764897.07 calls/second
    15-minute rate = 2751162.70 calls/second
               min = 613.00 nanoseconds
               max = 980.00 nanoseconds
              mean = 637.78 nanoseconds
            stddev = 21.83 nanoseconds
            median = 635.00 nanoseconds
              75% <= 639.00 nanoseconds
              95% <= 661.00 nanoseconds
              98% <= 681.00 nanoseconds
              99% <= 693.00 nanoseconds
            99.9% <= 969.00 nanoseconds

@artem-v artem-v merged commit 604cb97 into develop Jul 18, 2018
@artem-v artem-v deleted the feature/improve-benchmarks branch July 18, 2018 20:54
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
ready for review ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants