Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #285, Fix rate-limiting with cluster-mode #286

Merged

Conversation

YaacovHazan
Copy link
Collaborator

When a connection disconnected, the timer event was not free, and cause the test to keep running forever.

One of these cases is when we are starting the benchmark in cluster-mode and using the replica's ip/port.

When a connection disconnected, the timer event was not
free, and cause the test to keep running forever.

One of these cases is when we are starting the benchmark in
cluster-mode and using the replica's ip/port.
@filipecosta90 filipecosta90 self-requested a review January 5, 2025 19:09
@YaacovHazan YaacovHazan merged commit b3b3e4a into RedisLabs:master Jan 5, 2025
4 of 6 checks passed
@LINKIWI
Copy link

LINKIWI commented Jan 28, 2025

Hi, we believe that this commit may have severely regressed cluster mode performance in memtier_benchmark. After upgrading from 2.1.1 to 2.1.3, our performance tests are indicating a nearly 50% drop in throughput and 10x higher P99.9. This PR appears to be associated with the only commit in 2.1.3 that meaningfully changes the benchmark's core logic.

@filipecosta90
Copy link
Collaborator

@LINKIWI we have identified #294 as the root cause, and have a PR to fix it #295.

Altogether, that will revert the regression introduced in v2.1.1 and also reduced by 22% the CPU usage for the same QPS (even a bit higher). Can you check if that PR will fix your regression while we're reviewing it?

@LINKIWI
Copy link

LINKIWI commented Jan 28, 2025

Confirming that #295 applied on top of 2.1.3 indeed fixes the perf regression we are observing. Thanks! (Apologies for jumping to conclusions on the root cause; we didn't have time yesterday to dive deeper after noticing the original issue.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants