Add a new SQLiteBucket subclass that uses py-filelock #68
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #67 and #60
The changes in #61 made
SQLiteBucket
safe to use with multiple threads, and multiple processes with a sharedLimiter
.However, to work with multiple processes without a shared
Limiter
, a different means of inter-process communication is needed. A file lock seems to be the best way to do this. This does reduce performance, and also requires an extra dependency, so I'm making a separate bucket class for this.Changes
FileLockSQLiteBucket
Limiter
LimitContextDecorator
log messages down to debug, as it gets a bit spammy during the longer tests (withpytest -v -s
)poetry.dependencies
andpoetry.extras
poetry install -E all
orpip install pyrate-limiter[all]