Skip to content

google-cloud-datastore no longer works with freezegun #3284

Closed
@remcohaszing

Description

As of google-cloud-datastore 1.0.0 it no longer works together with freezegun.

Workaround: downgrade to google-cloud-datastore 0.23.0

  1. OS type and version:

Ubuntu 17.04, but it was also reproduced on 16.04.

  1. Python version and virtual environment information python --version
$ python --version
Python 3.6.0+
  1. google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze
$ pip list | grep google-cloud
gapic-google-cloud-datastore-v1 0.15.3 
google-cloud-core               0.24.0 
google-cloud-datastore          1.0.0  
google-cloud-storage            1.0.0  
proto-google-cloud-datastore-v1 0.90.3
  1. Stacktrace if available
py.test --tb native
Traceback (most recent call last):
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/retry.py", line 120, in inner
    return to_call(*args)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/retry.py", line 68, in inner
    return a_func(*updated_args, **kwargs)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/grpc/_channel.py", line 507, in __call__
    return _end_unary_response_blocking(state, call, False, deadline)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/grpc/_channel.py", line 455, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.INTERNAL, Received RST_STREAM with error code 2)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py", line 74, in _catch_remap_gax_error
    yield
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py", line 173, in commit
    return super(GAPICDatastoreAPI, self).commit(*args, **kwargs)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/gapic/datastore/v1/datastore_client.py", line 345, in commit
    return self._commit(request, options)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py", line 419, in inner
    return api_caller(api_call, this_settings, request)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py", line 407, in base_caller
    return api_call(*args)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py", line 368, in inner
    return a_func(*args, **kwargs)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/retry.py", line 126, in inner
    ' classified as transient', exception)
google.gax.errors.RetryError: GaxError(Exception occurred in retry method that was not classified as transient, caused by <_Rendezvous of RPC that terminated with (StatusCode.INTERNAL, Received RST_STREAM with error code 2)>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/remco/example/test.py", line 16, in test_put_frozen
    client.put(entity)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/client.py", line 384, in put
    self.put_multi(entities=[entity])
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/client.py", line 411, in put_multi
    current.commit()
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/batch.py", line 273, in commit
    self._commit()
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/batch.py", line 249, in _commit
    self.project, mode, self._mutations, transaction=self._id)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py", line 173, in commit
    return super(GAPICDatastoreAPI, self).commit(*args, **kwargs)
  File "/usr/lib/python3.6/contextlib.py", line 100, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py", line 82, in _catch_remap_gax_error
    six.reraise(error_class, new_exc, sys.exc_info()[2])
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py", line 74, in _catch_remap_gax_error
    yield
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py", line 173, in commit
    return super(GAPICDatastoreAPI, self).commit(*args, **kwargs)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/gapic/datastore/v1/datastore_client.py", line 345, in commit
    return self._commit(request, options)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py", line 419, in inner
    return api_caller(api_call, this_settings, request)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py", line 407, in base_caller
    return api_call(*args)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py", line 368, in inner
    return a_func(*args, **kwargs)
  File "/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/retry.py", line 126, in inner
    ' classified as transient', exception)
google.cloud.exceptions.InternalServerError: 500 Received RST_STREAM with error code 2
  1. Steps to reproduce

Decorate a function using freezegun.freeze_time from freezegun and put a gcloud datastore entity.

  1. Code example
import google.cloud.datastore
import pytest
from freezegun import freeze_time


# Works
def test_put():
    client = google.cloud.datastore.Client()
    entity = google.cloud.datastore.Entity(key=client.key('spam'))
    client.put(entity)


# Fails
@freeze_time('3216-08-04')
def test_put_frozen():
    client = google.cloud.datastore.Client()
    entity = google.cloud.datastore.Entity(key=client.key('spam'))
    client.put(entity)

The datastore was launched in the background using the following code:

def start_datastore():
    """
    Start an in memory gcloud datastore emulator.

    The datastore and all of its subprocesses are killed when the Python
    interpreter is shut down.

    Raises:
        RuntimeError: If the datastore failed to start.

    """
    tmpdir = tempfile.TemporaryDirectory()
    atexit.register(partial(tmpdir.cleanup))
    command = (
        'gcloud',
        'beta',
        'emulators',
        'datastore',
        'start',
        '--project', os.environ['GOOGLE_CLOUD_PROJECT'],
        '--no-store-on-disk',
        '--data-dir', tmpdir.name,
        '--consistency', '1.0'
    )
    proc = subprocess.Popen(
        command,
        stderr=subprocess.PIPE,
        preexec_fn=os.setsid)
    atexit.register(partial(os.killpg, os.getpgid(proc.pid), signal.SIGTERM))
    logs = []
    for lineno, line in enumerate(proc.stderr):
        line = line.decode()
        logs.append(line)
        if 'Dev App Server is now running' in line:
            return tmpdir
        # At the time of writing, 22 lines were printed before the
        # server was up and running. A small buffer is allowed in case
        # changes are introduced in the logging output.
        if lineno > 40:
            break
    raise RuntimeError('Unable to start datastore\n' + ''.join(logs))

Metadata

Assignees

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions