Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Optimization] boltdb readpath: reduce the code block affected by the lock tablesMtx in the TableManager.query() function #5829

Merged
merged 1 commit into from
Apr 8, 2022

Conversation

liguozhong
Copy link
Contributor

What this PR does / why we need it:
[Optimization] boltdb: reduce the code block affected by the lock tablesMtx in the TableManager.query() function.

util.DoParallelQueries(ctx, table, queries, callback) , free lock.
@sandeepsukhani boltdb issue.
image

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Checklist

  • Documentation added
  • Tests updated
  • Add an entry in the CHANGELOG.md about the changes.

…lesMtx in the TableManager.query() function.
@liguozhong liguozhong requested a review from a team as a code owner April 8, 2022 09:38
Copy link
Contributor

@sandeepsukhani sandeepsukhani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! The changes look good to me but I doubt it would help much with the write latency since the read path is using a shared lock.

@liguozhong liguozhong changed the title [Optimization] boltdb: reduce the code block affected by the lock tablesMtx in the TableManager.query() function [Optimization] boltdb readpath: reduce the code block affected by the lock tablesMtx in the TableManager.query() function Apr 8, 2022
@sandeepsukhani
Copy link
Contributor

@liguozhong this PR might help somewhat #5830

@liguozhong
Copy link
Contributor Author

Thanks for the PR! The changes look good to me but I doubt it would help much with the write latency since the read path is using a shared lock.

Thanks for the PR! The changes look good to me but I doubt it would help much with the write latency since the read path is using a shared lock.

I just started reading boltdb code, trying to find lock related code, check if there are optimized code blocks.
This pr is because I saw it during the viewing process.
I'm still trying to find performance optimization points for writes.
We're working on switching from Cassandra to boltdb-shipper, which is a huge change for us, and we hope our team will be more familiar with the boltdb-shipper part of the code before switching to boltdb-shipper from cassandra.

@liguozhong
Copy link
Contributor Author

liguozhong commented Apr 8, 2022

@liguozhong this PR might help somewhat #5830

👍,The code of this PR(https://github.com/grafana/loki/pull/5830) would be helpful. After the PR is merged, we will release boltdb-shipper again to observe the optimization effect.

@sandeepsukhani sandeepsukhani merged commit 1f8fc65 into grafana:main Apr 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants