Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the batch efficiency of kv client #55206

Closed
zyguan opened this issue Aug 6, 2024 · 0 comments · Fixed by #55383
Closed

Improve the batch efficiency of kv client #55206

zyguan opened this issue Aug 6, 2024 · 0 comments · Fixed by #55383
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@zyguan
Copy link
Contributor

zyguan commented Aug 6, 2024

Enhancement

Currently we use grpc stream to send batch commands requests. The grpc send operations just put the encoded dataframe to the internal buffer and return quickly (there is a seperate loopyWriter goroutine for flushing the buffered data). Thus it's seldom for the kv client to batch more than 1 request, the average batch size of kv requests is typically 1.1 ~ 1.2 even for a high throughput workload like sysbench oltp point select. Besides, we can only batch more requests when the tikv is detected as overload, this is also a rare event and it's disabled by default. The batch efficiency can be improved for these high throughput workloads to reduce the overhead of grpc and syscalls.

@zyguan zyguan added the type/enhancement The issue or PR belongs to an enhancement. label Aug 6, 2024
@ti-chi-bot ti-chi-bot bot closed this as completed in f9e9b14 Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant