Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(spanner): add support for Proto Columns #1084

Merged
merged 23 commits into from
May 16, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
7fd2f90
feat: Proto Columns Feature (#909)
harshachinta Mar 27, 2023
ee0bf5b
feat: Proto column feature tests and samples (#921)
harshachinta Apr 3, 2023
0eddbcf
feat: rever autogenerated code
harshachinta Jan 29, 2024
f44d51d
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jan 29, 2024
89d0e7d
Merge branch 'main' into proto-column-enhancement-alpha
harshachinta Jan 29, 2024
2f6f9d7
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jan 29, 2024
84d8ef6
fix: fix code
harshachinta Jan 29, 2024
b1bcf34
fix: fix code
harshachinta Jan 29, 2024
7bdfa28
Merge branch 'main' into proto-column-enhancement-alpha
harshachinta May 3, 2024
bfcb58d
fix(spanner): fix code
harshachinta May 3, 2024
5ff317c
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 3, 2024
02b3d5b
fix(spanner): skip emulator due to b/338557401
harshachinta May 3, 2024
01c3b2a
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 3, 2024
a7d60a1
fix(spanner): remove samples
harshachinta May 3, 2024
1fa1605
fix(spanner): update coverage
harshachinta May 3, 2024
f658ba0
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 3, 2024
f8e2fcc
Merge branch 'main' into proto-column-enhancement-alpha
harshachinta May 6, 2024
eea661f
chore(spanner): update coverage
harshachinta May 6, 2024
9a838e0
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 6, 2024
a76980c
fix(spanner): add samples and update proto schema
harshachinta May 11, 2024
19e97d1
fix(spanner): update samples database and emulator DDL
harshachinta May 13, 2024
60afda1
fix(spanner): update admin test to use autogenerated interfaces
harshachinta May 14, 2024
d050479
fix(spanner): comment refactoring
harshachinta May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
fix(spanner): comment refactoring
  • Loading branch information
harshachinta committed May 15, 2024
commit d0504793e032fd72b4ff081d8cd066266e33de21
8 changes: 7 additions & 1 deletion google/cloud/spanner_v1/_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,13 @@ def _parse_value_pb(value_pb, field_type, field_name, column_info=None):
:param field_name: column name

:type column_info: dict
:param column_info: (Optional) dict of column name and column information
:param column_info: (Optional) dict of column name and column information.
An object where column names as keys and custom objects as corresponding
values for deserialization. It's specifically useful for data types like
protobuf where deserialization logic is on user-specific code. When provided,
the custom object enables deserialization of backend-received column data.
If not provided, data remains serialized as bytes for Proto Messages and
integer for Proto Enums.

:rtype: varies on field_type
:returns: value extracted from value_pb
Expand Down
16 changes: 14 additions & 2 deletions google/cloud/spanner_v1/session.py
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,13 @@ def read(self, table, columns, keyset, index="", limit=0, column_info=None):
:param limit: (Optional) maximum number of rows to return

:type column_info: dict
:param column_info: (Optional) dict of mapping between column names and additional column information
:param column_info: (Optional) dict of mapping between column names and additional column information.
An object where column names as keys and custom objects as corresponding
values for deserialization. It's specifically useful for data types like
protobuf where deserialization logic is on user-specific code. When provided,
the custom object enables deserialization of backend-received column data.
If not provided, data remains serialized as bytes for Proto Messages and
integer for Proto Enums.

:rtype: :class:`~google.cloud.spanner_v1.streamed.StreamedResultSet`
:returns: a result set instance which can be used to consume rows.
Expand Down Expand Up @@ -308,7 +314,13 @@ def execute_sql(
:param timeout: (Optional) The timeout for this request.

:type column_info: dict
:param column_info: (Optional) dict of mapping between column names and additional column information
:param column_info: (Optional) dict of mapping between column names and additional column information.
An object where column names as keys and custom objects as corresponding
values for deserialization. It's specifically useful for data types like
protobuf where deserialization logic is on user-specific code. When provided,
the custom object enables deserialization of backend-received column data.
If not provided, data remains serialized as bytes for Proto Messages and
integer for Proto Enums.

:rtype: :class:`~google.cloud.spanner_v1.streamed.StreamedResultSet`
:returns: a result set instance which can be used to consume rows.
Expand Down
16 changes: 14 additions & 2 deletions google/cloud/spanner_v1/snapshot.py
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,13 @@ def read(
or regions should be used for non-transactional reads or queries.

:type column_info: dict
:param column_info: (Optional) dict of mapping between column names and additional column information
:param column_info: (Optional) dict of mapping between column names and additional column information.
An object where column names as keys and custom objects as corresponding
values for deserialization. It's specifically useful for data types like
protobuf where deserialization logic is on user-specific code. When provided,
the custom object enables deserialization of backend-received column data.
If not provided, data remains serialized as bytes for Proto Messages and
integer for Proto Enums.

:rtype: :class:`~google.cloud.spanner_v1.streamed.StreamedResultSet`
:returns: a result set instance which can be used to consume rows.
Expand Down Expand Up @@ -407,7 +413,13 @@ def execute_sql(
or regions should be used for non-transactional reads or queries.

:type column_info: dict
:param column_info: (Optional) dict of mapping between column names and additional column information
:param column_info: (Optional) dict of mapping between column names and additional column information.
An object where column names as keys and custom objects as corresponding
values for deserialization. It's specifically useful for data types like
protobuf where deserialization logic is on user-specific code. When provided,
the custom object enables deserialization of backend-received column data.
If not provided, data remains serialized as bytes for Proto Messages and
integer for Proto Enums.

:raises ValueError:
for reuse of single-use snapshots, or if a transaction ID is
Expand Down
12 changes: 10 additions & 2 deletions samples/samples/snippets.py
Original file line number Diff line number Diff line change
Expand Up @@ -3356,6 +3356,12 @@ def query_data_with_proto_types_parameter(instance_id, database_id):
"country": param_types.STRING,
"singerGenre": param_types.ProtoEnum(singer_pb2.Genre),
},
# column_info is an optional parameter and is used to deserialize
# the proto message and enum object back from bytearray and
# int respectively.
# If column_info is not passed for proto messages and enums, then
# the data types for these columns will be bytes and int
# respectively.
column_info={
"SingerInfo": singer_pb2.SingerInfo(),
"SingerInfoArray": singer_pb2.SingerInfo(),
Expand All @@ -3366,8 +3372,10 @@ def query_data_with_proto_types_parameter(instance_id, database_id):

for row in results:
print(
"SingerId: {}, SingerInfo: {}, SingerGenre: {}, "
"SingerInfoArray: {}, SingerGenreArray: {}".format(*row)
"SingerId: {}, SingerInfo: {}, SingerInfoNationality: {}, "
"SingerInfoArray: {}, SingerGenre: {}, SingerGenreArray: {}".format(
*row
)
)
# [END spanner_query_with_proto_types_parameter]

Expand Down
2 changes: 0 additions & 2 deletions tests/_fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,6 @@
) PRIMARY KEY (CartId);
"""

# TODO: Add Proto Bundle DDL statement in EMULATOR_DDL once b/338557401
# is fixed.
EMULATOR_DDL = """\
CREATE TABLE contacts (
contact_id INT64,
Expand Down