DISCUSSION: Switch to `apitools` as a dependency

[dhermes]

This was an issue before apitools was on PyPI and before it's dependencies were Py3 compatible. Now neither of these are blockers.

See #754 for context.

[dhermes]

I've tentatively started to test how python3-ready the apitools codebase is and haven't had much luck.

I couldn't even get tox to run: google/apitools#4

In addition, it seems there are some Python 2 specific imports, for example the StringIO module is used.

[dhermes]

@craigcitro Can we chat about this?

[dhermes]

@craigcitro Can you provide a snippet or two for

1. How to create and use a generated client
2. Compatibility / dependency issues to worry about (e.g. Py3)

[craigcitro]

1. using apitools is two steps: first, create a client, and second, load that code, then instantiate and call.

from scratch:

(sample)[~/t/storage_example] $ pip install google-apitools[cli]
... <installing> ...
(sample)[~/t/storage_example] $ gen_client --discovery_url=storage.v1 --nogenerate_cli --outdir=storage client
(sample)[~/t/storage_example] $ l
total 0
0 storage/
(sample)[~/t/storage_example] $ cd storage/
(sample)[~/t/storage_example/storage] $ l
total 120
  4 __init__.py              44 storage_v1_client.py     72 storage_v1_messages.py
(sample)[~/t/storage_example/storage] $ python
Python 2.7.9 (default, Jan  8 2015, 10:14:19)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import apitools.base.py as apitools_base
>>> import storage_v1_client as storage_client
>>> import storage_v1_messages as storage_messages
>>> client = storage_client.StorageV1()
>>> client.buckets.List(storage_messages.StorageBucketsListRequest(project='google.com:apitools'))
<Buckets
 items: [<Bucket
    ... copious output ...
 kind: u'storage#buckets'>
>>> buckets = _
>>> [bucket.name for bucket in buckets.items]
['apitools', ... <a zillion other buckets> ..., 'apitools-public']
>>> import StringIO
>>> upload = apitools_base.Upload.FromStream(StringIO.StringIO('hello world\n'), mime_type='text/plain')
>>> request = storage_messages.StorageObjectsInsertRequest()
>>> request.bucket = 'apitools'
>>> request.name = 'test_object'
>>> client.objects.Insert(request, upload=upload)
<new object metadata comes back>
>>> client.objects.Get(storage_messages.StorageObjectsGetRequest(bucket='apitools', object='test_object'), download=apitools_base.Download.FromFile('important.txt'))
Received bytes 0-11/12
Download complete
>>>
(sample)[~/t/storage_example/storage] $ cat important.txt
hello world
(sample)[~/t/storage_example/storage] $

there's plenty more in there -- i'm happy to do a little demo, if there's interest.

1. for compatibility, there are really three questions: generated clients, generated CLIs, and the gen_client tool itself. you should only care about the first, and eventually the third.

the first should already work in py3, or if not, i want to fix it. generated CLIs are furthest out, because it involves changing the codegen to not use gflags/apputils, but i don't think you care right now. for gen_client, it's actually a quick fix (where i plan to clean up 1-2 other small things), but i'm short on time (cf the long response on this! 😏)

[dhermes]

@craigcitro I'll take you up on that demo.

It seems the thorniest parts will be

• auth
• media/multipart/resumable uploads
• batching

[craigcitro]

sure -- thursday would be best for me, but we can coordinate by email. some offhand answers:

• for auth, you can either let apitools do it, or just pass a valid oauth2client.Credentials object as the credentials argument to the constructor, and it'll use those credentials.
• multipart and resumable are both handled for you; we choose a default based on size, but you can change it on the Upload/Download instance as desired.
• batching works fine (though, as discussed, not with media).

[dhermes]

@craigcitro RE: "(though, as discussed, not with media)." When did we discuss? Link? Media here means any files (i.e. non-metadata) at all or "media uploads"? Is it just a missing feature of the library or not supported by the backend?

[craigcitro]

@dhermes #654 (comment)

media means anything involving upload/download of files. apitools doesn't support it (and throws an appropriate error, i believe), but there's no support on the backend.