Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drive cache #296

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Drive cache #296

wants to merge 6 commits into from

Conversation

ixs
Copy link
Contributor

@ixs ixs commented Aug 13, 2020

Proposed change

Drive by default caches directory entries and will only invalidate them if a new directory is fetched.
This caching is happening in get_children().

Rework the cache to use a real TTL cache using the external cachetools.
Drive-By: Remove upper limits of dependency versions for click and keychain.
Fixes: #289
Fixes: #290

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New service (thank you!)
  • New feature (which adds functionality to an existing service)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests
  • Documentation or code sample

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

  • Documentation added/updated to README

Add cachetools to the requirements list.
Drive-By: Remove upper version limit for click, keyring and keyring.alt.
Fixes: picklepete#289
Fixes: picklepete#290
This is part one of a two part commit.
This part removes the existing permanent cache for folder content and
always fetch fresh data and return that.
The second part will add a real TTL'd cache.
Allow for a TTL expiry cache in Drive.
Cache is active by default, but can be changed on demand.
Add debug logging.
Allow specific functions to invalidate the cache if needed.
Makes sense after e.g. mkdir or an upload.
@ixs
Copy link
Contributor Author

ixs commented Aug 13, 2020

Hmm. Looks like cachetools does not exist for Python <3.5...
This means Py27 and Py34 are failing tests. Is this a problem or should we just drop support for those versions in master?

There's also a complain about me accessing _cache from outside the DriveService object... That is true, but I kinda feel it's acceptable to use the service-wide cache from the DriveNode object... But it is still a drive-internal cache which is why I'm prefixing it with an underscore.
What's the best solution here? Just silence pylint using # pylint: disable=W0212? Or should the code be reworked?

@Quentame
Copy link
Collaborator

Will remove support <3.6 or even <3.7

Coming to this PR just after this removal.

@Quentame
Copy link
Collaborator

Needs #303

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants