Invalid format in BigQuery destination dataset in request

[Sheshgiriv]

Environment details

• OS type and version: Mac OS Monterry 12.4
• Python version: Python 3.9.13
• pip version: pip 22.0.4 from /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pip (python 3.9)
• google-cloud-asset version: pip show google-cloud-asset
  Name: google-cloud-asset
  Version: 3.9.1
  Summary: Cloud Asset API API client library
  Home-page: https://github.com/googleapis/python-asset
  Author: Google LLC
  Author-email: googleapis-packages@google.com
  License: Apache 2.0
  Location: /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages
  Requires: google-api-core, google-cloud-access-context-manager, google-cloud-org-policy, google-cloud-os-config, grpc-google-iam-v1, proto-plus, protobuf
  Required-by:

Steps to reproduce

Trying to run the below python code snippet to export assets data to bigquery dataset

Code example

from google.cloud import asset_v1

project_id = 'my-project-id'
dataset = 'my-dataset-id'
table = 'res_table'
content_type = asset_v1.ContentType.RESOURCE

client = asset_v1.AssetServiceClient()
parent = project_id
output_config = asset_v1.OutputConfig()
output_config.bigquery_destination.dataset = dataset
output_config.bigquery_destination.table = table
output_config.bigquery_destination.force = False
response = client.export_assets(
    request={
        "parent": parent,
        "content_type": content_type,
        "output_config": output_config
        }
)
print(response.result())

Stack trace

myhomedir python-code % python3 collector.py
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 50, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "Invalid format in BigQuery destination dataset in request."
	debug_error_string = "{"created":"@1656575459.813917000","description":"Error received from peer ipv4:142.250.71.10:443","file":"src/core/lib/surface/call.cc","file_line":967,"grpc_message":"Invalid format in BigQuery destination dataset in request.","grpc_status":3}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/myhomedir/python-code/collector.py", line 14, in <module>
    response = client.export_assets(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/cloud/asset_v1/services/asset_service/client.py", line 617, in export_assets
    response = rpc(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py", line 154, in __call__
    return wrapped_func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 52, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Invalid format in BigQuery destination dataset in request. [field_violations {
  field: "output_config.bigquery_destination.dataset"
  description: "Invalid format in BigQuery destination dataset in request."
}
]

Thanks!

[vam-google]

Hello @Sheshgiriv, please make sure that all of the values you are sending match the expected format. Specifically I see that you are setting parent = project_id, but in the most similar code sample snippet I could find it is specified as parent = "projects/{}".format(project_id) https://github.com/googleapis/python-asset/blob/473e133e2674bf6a5ae655fe67be7d79fed2d8e9/samples/snippets/quickstart_exportassets.py#L49

Hopefully changing that line (so that parent value is prefixed with projects/ can fix your issue.

Please let us know if that fixes issue for you.

[Sheshgiriv]

Hi @vam-google , I changed the code to below format even then the issue exists

parent = project_id to
parent = "projects/{}".format(project_id)

not able to provide the project id in the correct format gives the below exception

`myhomedir python-code % python3 collector.py
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 50, in error_remapped_callable
return callable_(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_channel.py", line 946, in call
return _end_unary_response_blocking(state, call, False, None)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.PERMISSION_DENIED
details = "The caller does not have permission"
debug_error_string = "{"created":"@1658318128.385897000","description":"Error received from peer ipv4:142.250.196.10:443","file":"src/core/lib/surface/call.cc","file_line":967,"grpc_message":"The caller does not have permission","grpc_status":7}"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/myhomedir/python-code/collector.py", line 14, in
response = client.export_assets(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/cloud/asset_v1/services/asset_service/client.py", line 617, in export_assets
response = rpc(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py", line 154, in call
return wrapped_func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 52, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.PermissionDenied: 403 The caller does not have permission`

[parthea]

I'm going to transfer this issue to the google-cloud-python repository as we are preparing to move the code for google-cloud-asset to that repository in the next 1-2 weeks.