Skip to content

Commit

Permalink
[tf.data / Bigtable] Update docs and method docstrings
Browse files Browse the repository at this point in the history
* Add link to updating scope on a running VM
* Add code formatting and Python syntax highlighting
* Clarify kwargs argument formatting
* Fix method name in docstring

PiperOrigin-RevId: 207204628
  • Loading branch information
mbrukman authored and tensorflower-gardener committed Aug 3, 2018
1 parent 0dbd7e3 commit 2d38196
Show file tree
Hide file tree
Showing 2 changed files with 27 additions and 20 deletions.
7 changes: 5 additions & 2 deletions tensorflow/contrib/bigtable/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -324,7 +324,7 @@ If you encounter a log line that includes the following:
"filename":"/usr/share/grpc/roots.pem"
```

you likely need to copy the [gRPC roots.pem file][grpcPem] to
you likely need to copy the [gRPC `roots.pem` file][grpcPem] to
`/usr/share/grpc/roots.pem` on your local machine.

[grpcPem]: https://github.com/grpc/grpc/blob/master/etc/roots.pem
Expand All @@ -338,7 +338,10 @@ are available.
- **Compute Engine**: When running on Compute Engine, the client will often use
the service account from the virtual machine's metadata service. Be sure to
authorize your Compute Engine VM to have access to the Cloud Bigtable service
when creating your VM.
when creating your VM, or [update the VM's scopes][update-vm-scopes] on a
running VM if you run into this issue.
- **Cloud TPU**: Your Cloud TPUs run with the designated Cloud TPU service
account dedicated to your GCP project. Ensure the service account has been
authorized via the Cloud Console to access your Cloud Bigtable instances.

[update-vm-scopes]: https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances#changeserviceaccountandscopes
40 changes: 22 additions & 18 deletions tensorflow/contrib/bigtable/python/ops/bigtable_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""The Python API for TensorFlow's Bigtable integration.
"""The Python API for TensorFlow's Cloud Bigtable integration.
TensorFlow has support for reading from and writing to Cloud Bigtable. To use
the Bigtable TensorFlow integration, first create a BigtableClient (which
configures your connection to Cloud Bigtable), and then open a Table. The Table
object then allows you to create numerous @{tf.data.Dataset}s to read data, or
write a @{tf.data.Dataset} object to the underlying Bigtable Table.
TensorFlow + Cloud Bigtable integration, first create a BigtableClient to
configure your connection to Cloud Bigtable, and then create a BigtableTable
object to allow you to create numerous @{tf.data.Dataset}s to read data, or
write a @{tf.data.Dataset} object to the underlying Cloud Bigtable table.
For background on Google Cloud Bigtable, see: https://cloud.google.com/bigtable.
For background on Cloud Bigtable, see: https://cloud.google.com/bigtable .
"""

from __future__ import absolute_import
Expand Down Expand Up @@ -48,7 +48,7 @@ class BigtableClient(object):
"""BigtableClient is the entrypoint for interacting with Cloud Bigtable in TF.
BigtableClient encapsulates a connection to Cloud Bigtable, and exposes the
`table` method to open a Bigtable Table.
`table` method to open a Bigtable table.
"""

def __init__(self,
Expand Down Expand Up @@ -94,16 +94,16 @@ def __init__(self,
project_id, instance_id, connection_pool_size, max_receive_message_size)

def table(self, name, snapshot=None):
"""Opens a table and returns a `BigtableTable` object.
"""Opens a table and returns a `tf.contrib.bigtable.BigtableTable` object.
Args:
name: A `tf.string` `tf.Tensor` name of the table to open.
snapshot: Either a `tf.string` `tf.Tensor` snapshot id, or `True` to
request the creation of a snapshot. (Note: currently unimplemented.)
Returns:
A `BigtableTable` python object representing the operations available on
the table.
A `tf.contrib.bigtable.BigtableTable` Python object representing the
operations available on the table.
"""
# TODO(saeta): Implement snapshot functionality.
table = gen_bigtable_ops.bigtable_table(self._resource, name)
Expand Down Expand Up @@ -133,7 +133,8 @@ def lookup_columns(self, *args, **kwargs):
"""Retrieves the values of columns for a dataset of keys.
Example usage:
```
```python
table = bigtable_client.table("my_table")
key_dataset = table.get_keys_prefix("imagenet")
images = key_dataset.apply(table.lookup_columns(("cf1", "image"),
Expand All @@ -144,23 +145,26 @@ def lookup_columns(self, *args, **kwargs):
Alternatively, you can use keyword arguments to specify the columns to
capture. Example (same as above, rewritten):
```
```python
table = bigtable_client.table("my_table")
key_dataset = table.get_keys_prefix("imagenet")
images = key_dataset.apply(table.lookup_columns(
cf1="image", cf2=("label", "boundingbox")))
training_data = images.map(parse_and_crop, num_parallel_calls=64).batch(128)
```
Note: certain kwargs keys are reserved, and thus some column families cannot
be identified using the kwargs syntax. Instead, please use the args syntax.
This list includes:
Note: certain `kwargs` keys are reserved, and thus, some column families
cannot be identified using the `kwargs` syntax. Instead, please use the
`args` syntax. This list includes:
- 'name'
This list can change at any time.
Note: this list can change at any time.
Args:
*args: A list of tuples containing (column family, column name) pairs.
**kwargs: Column families and
**kwargs: Column families (keys) and column qualifiers (values).
Returns:
A function that can be passed to `tf.data.Dataset.apply` to retrieve the
Expand Down Expand Up @@ -712,7 +716,7 @@ def _as_variant_tensor(self):


class _BigtableSampleKeyPairsDataset(dataset_ops.Dataset):
"""_BigtableKeyRangeDataset returns key pairs from the Bigtable.
"""_BigtableSampleKeyPairsDataset returns key pairs from a Bigtable table.
"""

def __init__(self, table, prefix, start, end):
Expand Down

0 comments on commit 2d38196

Please sign in to comment.