Merge pull request googleapis#1557 from dhermes/fix-1548-part2

Splitting Bigtable Row into 3 classes based on the mutation type / RPC method used in commit()

Author: dhermes

docs/bigtable-client-intro.rst

@@ -21,18 +21,19 @@ defaults
 However, you may over-ride them and these will be used throughout all API
 requests made with the ``client`` you create.
 
-Authorization
+Configuration
 -------------
 
 - For an overview of authentication in ``gcloud-python``,
   see :doc:`gcloud-auth`.
 
 - In addition to any authentication configuration, you can also set the
-  :envvar:`GCLOUD_PROJECT` environment variable for the project you'd like
-  to interact with. If you are Google App Engine or Google Compute Engine
-  this will be detected automatically. (Setting this environment
-  variable is not required, you may instead pass the ``project`` explicitly
-  when constructing a :class:`Client <gcloud.storage.client.Client>`).
+  :envvar:`GCLOUD_PROJECT` environment variable for the Google Cloud Console
+  project you'd like to interact with. If your code is running in Google App
+  Engine or Google Compute Engine the project will be detected automatically.
+  (Setting this environment variable is not required, you may instead pass the
+  ``project`` explicitly when constructing a
+  :class:`Client <gcloud.storage.client.Client>`).
 
 - After configuring your environment, create a
   :class:`Client <gcloud.storage.client.Client>`

docs/bigtable-cluster-api.rst

@@ -85,11 +85,13 @@ Check on Current Operation
 .. note::
 
     When modifying a cluster (via a `CreateCluster`_, `UpdateCluster`_ or
-    `UndeleteCluster`_ request), the Bigtable API will return a long-running
-    `Operation`_. This will be stored on the object after each of
+    `UndeleteCluster`_ request), the Bigtable API will return a
+    `long-running operation`_ and a corresponding
+    :class:`Operation <gcloud.bigtable.cluster.Operation>` object
+    will be returned by each of
     :meth:`create() <gcloud.bigtable.cluster.Cluster.create>`,
     :meth:`update() <gcloud.bigtable.cluster.Cluster.update>` and
-    :meth:`undelete() <gcloud.bigtable.cluster.Cluster.undelete>` are called.
+    :meth:`undelete() <gcloud.bigtable.cluster.Cluster.undelete>`.
 
 You can check if a long-running operation (for a
 :meth:`create() <gcloud.bigtable.cluster.Cluster.create>`,
@@ -176,4 +178,4 @@ Head next to learn about the :doc:`bigtable-table-api`.
 .. _ListClusters: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L44-L46
 .. _GetOperation: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/longrunning/operations.proto#L43-L45
 .. _UndeleteCluster: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L126-L128
-.. _Operation: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/longrunning/operations.proto#L73-L102
+.. _long-running operation: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/longrunning/operations.proto#L73-L102

docs/bigtable-data-api.rst

@@ -24,23 +24,11 @@ Cells vs. Columns vs. Column Families
 Modifying Data
 ++++++++++++++
 
-Since data is stored in cells, which are stored in rows, the
-:class:`Row <gcloud.bigtable.row.Row>` class is the only class used to
-modify (write, update, delete) data in a
+Since data is stored in cells, which are stored in rows, we
+use the metaphor of a **row** in classes that are used to modify
+(write, update, delete) data in a
 :class:`Table <gcloud.bigtable.table.Table>`.
 
-Row Factory
------------
-
-To create a :class:`Row <gcloud.bigtable.row.Row>` object
-
-.. code:: python
-
-    row = table.row(row_key)
-
-Unlike the previous string values we've used before, the row key must
-be ``bytes``.
-
 Direct vs. Conditional vs. Append
 ---------------------------------
 
@@ -49,43 +37,65 @@ There are three ways to modify data in a table, described by the
 methods.
 
 * The **direct** way is via `MutateRow`_ which involves simply
-  adding, overwriting or deleting cells.
+  adding, overwriting or deleting cells. The
+  :class:`DirectRow <gcloud.bigtable.row.DirectRow>` class
+  handles direct mutations.
 * The **conditional** way is via `CheckAndMutateRow`_. This method
   first checks if some filter is matched in a a given row, then
   applies one of two sets of mutations, depending on if a match
   occurred or not. (These mutation sets are called the "true
-  mutations" and "false mutations".)
+  mutations" and "false mutations".) The
+  :class:`ConditionalRow <gcloud.bigtable.row.ConditionalRow>` class
+  handles conditional mutations.
 * The **append** way is via `ReadModifyWriteRow`_. This simply
   appends (as bytes) or increments (as an integer) data in a presumed
-  existing cell in a row.
+  existing cell in a row. The
+  :class:`AppendRow <gcloud.bigtable.row.AppendRow>` class
+  handles append mutations.
 
-Building Up Mutations
----------------------
+Row Factory
+-----------
 
-In all three cases, a set of mutations (or two sets) are built up
-on a :class:`Row <gcloud.bigtable.row.Row>` before they are sent of
-in a batch via :meth:`commit() <gcloud.bigtable.row.Row.commit>`:
+A single factory can be used to create any of the three row types.
+To create a :class:`DirectRow <gcloud.bigtable.row.DirectRow>`:
 
 .. code:: python
 
-    row.commit()
+    row = table.row(row_key)
 
-To send **append** mutations in batch, use
-:meth:`commit_modifications() <gcloud.bigtable.row.Row.commit_modifications>`:
+Unlike the previous string values we've used before, the row key must
+be ``bytes``.
+
+To create a :class:`ConditionalRow <gcloud.bigtable.row.ConditionalRow>`,
+first create a :class:`RowFilter <gcloud.bigtable.row.RowFilter>` and
+then
 
 .. code:: python
 
-    row.commit_modifications()
+    cond_row = table.row(row_key, filter_=filter_)
 
-We have a small set of methods on the :class:`Row <gcloud.bigtable.row.Row>`
-to build these mutations up.
+To create an :class:`AppendRow <gcloud.bigtable.row.AppendRow>`
+
+.. code:: python
+
+    append_row = table.row(row_key, append=True)
+
+Building Up Mutations
+---------------------
+
+In all three cases, a set of mutations (or two sets) are built up
+on a row before they are sent of in a batch via
+
+.. code:: python
+
+    row.commit()
 
 Direct Mutations
 ----------------
 
 Direct mutations can be added via one of four methods
 
-* :meth:`set_cell() <gcloud.bigtable.row.Row.set_cell>` allows a
+* :meth:`set_cell() <gcloud.bigtable.row.DirectRow.set_cell>` allows a
   single value to be written to a column
 
   .. code:: python
@@ -97,9 +107,9 @@ Direct mutations can be added via one of four methods
   Bigtable server will be used when the cell is stored.
 
   The value can either by bytes or an integer (which will be converted to
-  bytes as an unsigned 64-bit integer).
+  bytes as a signed 64-bit integer).
 
-* :meth:`delete_cell() <gcloud.bigtable.row.Row.delete_cell>` deletes
+* :meth:`delete_cell() <gcloud.bigtable.row.DirectRow.delete_cell>` deletes
   all cells (i.e. for all timestamps) in a given column
 
   .. code:: python
@@ -117,8 +127,9 @@ Direct mutations can be added via one of four methods
       row.delete_cell(column_family_id, column,
                       time_range=time_range)
 
-* :meth:`delete_cells() <gcloud.bigtable.row.Row.delete_cells>` does
-  the same thing as :meth:`delete_cell() <gcloud.bigtable.row.Row.delete_cell>`
+* :meth:`delete_cells() <gcloud.bigtable.row.DirectRow.delete_cells>` does
+  the same thing as
+  :meth:`delete_cell() <gcloud.bigtable.row.DirectRow.delete_cell>`
   but accepts a list of columns in a column family rather than a single one.
 
   .. code:: python
@@ -127,15 +138,16 @@ Direct mutations can be added via one of four methods
                        time_range=time_range)
 
   In addition, if we want to delete cells from every column in a column family,
-  the special :attr:`ALL_COLUMNS <gcloud.bigtable.row.Row.ALL_COLUMNS>` value
-  can be used
+  the special :attr:`ALL_COLUMNS <gcloud.bigtable.row.DirectRow.ALL_COLUMNS>`
+  value can be used
 
   .. code:: python
 
-      row.delete_cells(column_family_id, Row.ALL_COLUMNS,
+      row.delete_cells(column_family_id, row.ALL_COLUMNS,
                        time_range=time_range)
 
-* :meth:`delete() <gcloud.bigtable.row.Row.delete>` will delete the entire row
+* :meth:`delete() <gcloud.bigtable.row.DirectRow.delete>` will delete the
+  entire row
 
   .. code:: python
 
@@ -145,57 +157,42 @@ Conditional Mutations
 ---------------------
 
 Making **conditional** modifications is essentially identical
-to **direct** modifications, but we need to specify a filter to match
-against in the row:
+to **direct** modifications: it uses the exact same methods
+to accumulate mutations.
 
-.. code:: python
-
-    row = table.row(row_key, filter_=filter_val)
-
-See the :class:`Row <gcloud.bigtable.row.Row>` class for more information
-about acceptable values for ``filter_``.
-
-The only other difference from **direct** modifications are that each mutation
-added must specify a ``state``: will the mutation be applied if the filter
-matches or if it fails to match.
+However, each mutation added must specify a ``state``: will the mutation be
+applied if the filter matches or if it fails to match.
 
 For example:
 
 .. code:: python
 
-    row.set_cell(column_family_id, column, value,
-                 timestamp=timestamp, state=True)
+    cond_row.set_cell(column_family_id, column, value,
+                      timestamp=timestamp, state=True)
 
 will add to the set of true mutations.
 
-.. note::
-
-    If ``state`` is passed when no ``filter_`` is set on a
-    :class:`Row <gcloud.bigtable.row.Row>`, adding the mutation will fail.
-    Similarly, if no ``state`` is passed when a ``filter_`` has been set,
-    adding the mutation will fail.
-
 Append Mutations
 ----------------
 
 Append mutations can be added via one of two methods
 
-* :meth:`append_cell_value() <gcloud.bigtable.row.Row.append_cell_value>`
+* :meth:`append_cell_value() <gcloud.bigtable.row.AppendRow.append_cell_value>`
   appends a bytes value to an existing cell:
 
   .. code:: python
 
-      row.append_cell_value(column_family_id, column, bytes_value)
+      append_row.append_cell_value(column_family_id, column, bytes_value)
 
-* :meth:`increment_cell_value() <gcloud.bigtable.row.Row.increment_cell_value>`
+* :meth:`increment_cell_value() <gcloud.bigtable.row.AppendRow.increment_cell_value>`
   increments an integer value in an existing cell:
 
   .. code:: python
 
-      row.increment_cell_value(column_family_id, column, int_value)
+      append_row.increment_cell_value(column_family_id, column, int_value)
 
   Since only bytes are stored in a cell, the cell value is decoded as
-  an unsigned 64-bit integer before being incremented. (This happens on
+  a signed 64-bit integer before being incremented. (This happens on
   the Google Cloud Bigtable server, not in the library.)
 
 Notice that no timestamp was specified. This is because **append** mutations
@@ -208,18 +205,10 @@ Starting Fresh
 --------------
 
 If accumulated mutations need to be dropped, use
-:meth:`clear_mutations() <gcloud.bigtable.row.Row.clear_mutations>`
-
-.. code:: python
-
-    row.clear_mutations()
-
-To clear **append** mutations, use
-:meth:`clear_modification_rules() <gcloud.bigtable.row.Row.clear_modification_rules>`
 
 .. code:: python
 
-    row.clear_modification_rules()
+    row.clear()
 
 Reading Data
 ++++++++++++
@@ -260,19 +249,20 @@ To make a `ReadRows`_ API request for a single row key, use
     >>> cell.timestamp
     datetime.datetime(2016, 2, 27, 3, 41, 18, 122823, tzinfo=<UTC>)
 
-Rather than returning a :class:`Row <gcloud.bigtable.row.Row>`, this method
-returns a :class:`PartialRowData <gcloud.bigtable.row_data.PartialRowData>`
+Rather than returning a :class:`DirectRow <gcloud.bigtable.row.DirectRow>`
+or similar class, this method returns a
+:class:`PartialRowData <gcloud.bigtable.row_data.PartialRowData>`
 instance. This class is used for reading and parsing data rather than for
-modifying data (as :class:`Row <gcloud.bigtable.row.Row>` is).
+modifying data (as :class:`DirectRow <gcloud.bigtable.row.DirectRow>` is).
 
-A filter can also be applied to the
+A filter can also be applied to the results:
 
 .. code:: python
 
     row_data = table.read_row(row_key, filter_=filter_val)
 
 The allowable ``filter_`` values are the same as those used for a
-:class:`Row <gcloud.bigtable.row.Row>` with **conditional** mutations. For
+:class:`ConditionalRow <gcloud.bigtable.row.ConditionalRow>`. For
 more information, see the
 :meth:`Table.read_row() <gcloud.bigtable.table.Table.read_row>` documentation.
 

docs/bigtable-row.rst

@@ -2,7 +2,8 @@ Bigtable Row
 ============
 
 It is possible to use a :class:`RowFilter <gcloud.bigtable.row.RowFilter>`
-when adding mutations to a :class:`Row <gcloud.bigtable.row.Row>` and when
+when adding mutations to a
+:class:`ConditionalRow <gcloud.bigtable.row.ConditionalRow>` and when
 reading row data with :meth:`read_row() <gcloud.bigtable.table.Table.read_row>`
 :meth:`read_rows() <gcloud.bigtable.table.Table.read_rows>`.
 

gcloud/bigtable/happybase/table.py

@@ -603,12 +603,12 @@ def counter_inc(self, row, column, value=1):
         :rtype: int
         :returns: Counter value after incrementing.
         """
-        row = self._low_level_table.row(row)
+        row = self._low_level_table.row(row, append=True)
         if isinstance(column, six.binary_type):
             column = column.decode('utf-8')
         column_family_id, column_qualifier = column.split(':')
         row.increment_cell_value(column_family_id, column_qualifier, value)
-        # See row.commit_modifications() will return a dictionary:
+        # See AppendRow.commit() will return a dictionary:
         # {
         #     u'col-fam-id': {
         #         b'col-name1': [
@@ -618,7 +618,7 @@ def counter_inc(self, row, column, value=1):
         #         ...
         #     },
         # }
-        modified_cells = row.commit_modifications()
+        modified_cells = row.commit()
         # Get the cells in the modified column,
         column_cells = modified_cells[column_family_id][column_qualifier]
         # Make sure there is exactly one cell in the column.

gcloud/bigtable/happybase/test_table.py

@@ -871,10 +871,12 @@ def _counter_inc_helper(self, row, column, value, commit_result):
         table = self._makeOne(name, connection)
         # Mock the return values.
         table._low_level_table = _MockLowLevelTable()
-        table._low_level_table.row_values[row] = _MockLowLevelRow(
+        table._low_level_table.row_values[row] = row_obj = _MockLowLevelRow(
             row, commit_result=commit_result)
 
+        self.assertFalse(row_obj._append)
         result = table.counter_inc(row, column, value=value)
+        self.assertTrue(row_obj._append)
 
         incremented_value = value + _MockLowLevelRow.COUNTER_DEFAULT
         self.assertEqual(result, incremented_value)
@@ -1431,8 +1433,10 @@ def list_column_families(self):
         self.list_column_families_calls += 1
         return self.column_families
 
-    def row(self, row_key):
-        return self.row_values[row_key]
+    def row(self, row_key, append=None):
+        result = self.row_values[row_key]
+        result._append = append
+        return result
 
     def read_row(self, *args, **kwargs):
         self.read_row_calls.append((args, kwargs))
@@ -1449,6 +1453,7 @@ class _MockLowLevelRow(object):
 
     def __init__(self, row_key, commit_result=None):
         self.row_key = row_key
+        self._append = False
         self.counts = {}
         self.commit_result = commit_result
 
@@ -1457,7 +1462,7 @@ def increment_cell_value(self, column_family_id, column, int_value):
                                        self.COUNTER_DEFAULT)
         self.counts[(column_family_id, column)] = count + int_value
 
-    def commit_modifications(self):
+    def commit(self):
         return self.commit_result