Incompatible with CPP protobuf implementation for Python

[junghoahnsc]

Hello,

I'm trying to use C++ protobuf implementation for Python since pure Python proto is too slow.
But importing gcloud.bigtable package throws an exception TypeError: Descriptors should not be created directly, but only retrieved from their parent..

Is there any way to fix this problem?

Thanks,

1. OS type and version
   Tested on ubuntu 14.04 / osx 10.12.

2. Python version and virtual environment information python --version
   Python 2.7.6

3. google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze
   google-cloud 0.22.0 / 0.18.3

4. Stacktrace if available

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jungho.ahn/pyproto/protobuf/xx/local/lib/python2.7/site-packages/google/cloud/bigtable/__init__.py", line 18, in <module>
    from google.cloud.bigtable.client import Client
  File "/home/jungho.ahn/pyproto/protobuf/xx/local/lib/python2.7/site-packages/google/cloud/bigtable/client.py", line 33, in <module>
    from google.longrunning import operations_grpc
  File "/home/jungho.ahn/pyproto/protobuf/xx/local/lib/python2.7/site-packages/google/longrunning/operations_grpc.py", line 47, in <module>
    options=None),
  File "/home/jungho.ahn/pyproto/protobuf/python/google/protobuf/descriptor.py", line 494, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors should not be created directly, but only retrieved from their parent.

5. Steps to reproduce

// Install CPP proto implementation.
$ git clone https://github.com/google/protobuf.git
$ cd protobuf
$ ./autogen.sh
$ CXXFLAGS="-fPIC -g -O2" ./configure
$ make -j12
$ export PROTOC=$PWD/src/protoc
$ cd python
$ python setup.py bdist_wheel --cpp_implementation --compile_static_extension
$ pip uninstall protobuf
$ pip install dist/<wheel file name> (e.g., protobuf-3.2.0-cp27-cp27mu-linux_x86_64.whl)

// Enable CPP proto implementation.
$ export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
$ python -c 'from gcloud import bigtable'

6. Code example
   from gcloud import bigtable

[dhermes]

@junghoahnsc Can you please try upgrading to google-cloud, the package moved from gcloud to google-cloud (a simple rename) after August 2016.

Unfortunately, I'm not sure the upgrade will fix the problem, but I want to make sure we're on the same page.

[junghoahnsc]

Yes, actually I tried to upgrade to google-cloud 0.22.0, but it showed the same issue.

[tseaver]

Note that the error is being raised from code in google.longrunning.operations_grpc, which is not maintained in this repository (it is auto-generated from the proto file), which means we won't be able to fix it here.

@bjwatson, @geigerj can you comment?

[dhermes]

@tseaver The error is actually in protobuf itself.

[junghoahnsc]

Should I file this issue to https://github.com/google/protobuf?

[dhermes]

@junghoahnsc Maybe? I'm interested if @bjwatson or @geigerj (or @lukesneeringer) have any insights.

[lukesneeringer]

I think you will need to file an issue there. I looked at the code and it's specifically nested within a if _USE_C_DESCRIPTORS:, and it is not obvious to me what it is supposed to be doing. Sorry. :-(

[dhermes]

@junghoahnsc Closing out since we can't really offer help. Please post back here if you aren't able to resolve?

[lukesneeringer]

This ended up being solved by this pull request in googleapis-common-protos.