Skip to content

Commit

Permalink
fix: Replace default file_cache usage with simple memory cache (#4184)
Browse files Browse the repository at this point in the history
* fix: Replace default file_cache usage with simple memory cache

The original main.py creates noisy log messages when the function is run file_cache is unavailable when using oauth2client >= 4.0.0:

```
2020-06-26T19:34:03.459Z datastore_export file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export Traceback (most recent call last): E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export   File "/env/local/lib/python3.7/site-packages/googleapiclient/discovery_cache/file_cache.py", line 33, in <module> E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export     from oauth2client.contrib.locked_file import LockedFile E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export ModuleNotFoundError: No module named 'oauth2client' E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export  E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export During handling of the above exception, another exception occurred: E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export  E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export Traceback (most recent call last): E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export   File "/env/local/lib/python3.7/site-packages/googleapiclient/discovery_cache/file_cache.py", line 37, in <module> E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export     from oauth2client.locked_file import LockedFile E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export ModuleNotFoundError: No module named 'oauth2client' E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export  E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export During handling of the above exception, another exception occurred: E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export  E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export Traceback (most recent call last): E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export   File "/env/local/lib/python3.7/site-packages/googleapiclient/discovery_cache/__init__.py", line 44, in autodetect E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export     from . import file_cache E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export   File "/env/local/lib/python3.7/site-packages/googleapiclient/discovery_cache/file_cache.py", line 41, in <module> E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export     "file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth" E  datastore_export 
2020-06-26T19:34:03.459Z datastore_export ImportError: file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth E  datastore_export 
```

While they don't interfere with the function's operation, they are disconcerting and not necessary. This implements the workaround suggested in googleapis/google-api-python-client#325 (which creates an in-memory cache object to use in replacement).

* Add explanatory comment

Add a comment explaining the inclusion of MemoryCache instance in the `build` invocation.

* Update main.py

PEP8-mandated two newlines between top-level elements

Co-authored-by: Christopher Wilcox <crwilcox@google.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
  • Loading branch information
3 people authored Jul 1, 2020
1 parent f69acf1 commit 22788da
Showing 1 changed file with 15 additions and 1 deletion.
16 changes: 15 additions & 1 deletion datastore/schedule-export/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,22 @@
import os

from googleapiclient.discovery import build
from googleapiclient.discovery_cache.base import Cache

datastore = build('datastore', 'v1')

class MemoryCache(Cache):
_CACHE = {}

def get(self, url):
return MemoryCache._CACHE.get(url)

def set(self, url, content):
MemoryCache._CACHE[url] = content


# The default cache (file_cache) is unavailable when using oauth2client >= 4.0.0 or google-auth,
# and it will log worrisome messages unless given another interface to use.
datastore = build('datastore', 'v1', cache=MemoryCache())
project_id = os.environ.get('GCP_PROJECT')


Expand Down

0 comments on commit 22788da

Please sign in to comment.