Fix broken tests. tox, travis, and py2/py3 #416
Conversation
|
A couple other changes got in here. Trying to reconcile a couple different changes. |
The hash seed on different python versions will produce different test results here unless we sort explicitly.
The file format changed and if you try to load a py2 cache on py3, it barfs. This changes our code to use different filenames for different python versions.
ralphbean
changed the title
|
And... they're passing again. 🍏 |
| os.getenv('XDG_CACHE_HOME', '~/.cache') + '/dagd.dbm') | ||
| PYVER = '%i.%i' % sys.version_info[:2] | ||
| DOGPILE_CACHE_PATH = os.path.expanduser(''.join([ | ||
| os.getenv('XDG_CACHE_HOME', '~/.cache'), '/dagd-py', PYVER, '.dbm'])) |
There was a problem hiding this comment.
Is the cache database really incompatible between python2/python3?
There was a problem hiding this comment.
As far as I can tell, yes. Will verify.
There was a problem hiding this comment.
The actual database format is different? That doesn't make much sense to me.
Or is it just a bytes-vs-strings issue where we could just convert bytes to unicode in python3?
There was a problem hiding this comment.
Well, here's some info:
# in python2
>>> import dogpile.cache
>>> cache = dogpile.cache.make_region().configure('dogpile.cache.dbm', arguments=dict(filename='foo.db'))
>>> cache.get_or_create('key', lambda: 'value')
'value'and then...
# in python3
>>> import dogpile.cache
>>> cache = dogpile.cache.make_region().configure('dogpile.cache.dbm', arguments=dict(filename='foo.db'))
>>> cache.get_or_create('key', lambda: 'value')
Traceback (most recent call last):
File "<input>", line 1, in <module>
cache.get_or_create('key', lambda: 'value')
File "/usr/lib/python3.5/site-packages/dogpile/cache/region.py", line 825, in get_or_create
async_creator) as value:
File "/usr/lib/python3.5/site-packages/dogpile/lock.py", line 154, in __enter__
return self._enter()
File "/usr/lib/python3.5/site-packages/dogpile/lock.py", line 87, in _enter
value = value_fn()
File "/usr/lib/python3.5/site-packages/dogpile/cache/region.py", line 780, in get_value
value = self.backend.get(key)
File "/usr/lib/python3.5/site-packages/dogpile/cache/backends/file.py", line 220, in get
with self._dbm_file(False) as dbm:
File "/usr/lib64/python3.5/contextlib.py", line 59, in __enter__
return next(self.gen)
File "/usr/lib/python3.5/site-packages/dogpile/cache/backends/file.py", line 215, in _dbm_file
"w" if write else "r")
File "/usr/lib64/python3.5/dbm/__init__.py", line 88, in open
raise error[0]("db type could not be determined")
dbm.error: db type could not be determined# Inspecting the file created by python2
~❯ file foo.db
foo.db: Berkeley DB (Hash, version 9, native byte-order)
# inspecting the file created by python3
~❯ file foo.db
foo.db: GNU dbm 1.x or ndbm database, little endian, 64-bitThere was a problem hiding this comment.
Ah ha, the python 3 docs point to this third party lib which explains that the berkeley db interface was included in py2.3 through py2.7 but is not included in python3. I'll see if just installing it will do as a workaround.
There was a problem hiding this comment.
Yeah, just installing pybsddb doesn't have any effect. dogpile.cache would have to invoke it explicitly and it doesn't look like it has a flag to tell it to do so.
There was a problem hiding this comment.
Ah ha, the python 3 docs point to this third party lib which explains that the berkeley db interface was included in py2.3 through py2.7 but is not included in python3.
I think including a comment to that effect here would be sufficient. Cache incompatibility isn't a problem, it's just surprising.
|
Thanks @ryneeverett! |
A number of changes here. See discussion in #406 as well as #384.