Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests.system.test_session_api: test_transaction_execute_sql_dml_returning failed #889

Closed
flaky-bot bot opened this issue Jan 24, 2023 · 1 comment
Closed
Assignees
Labels
api: spanner Issues related to the googleapis/python-spanner API. flakybot: flaky Tells the Flaky Bot not to close or comment on this issue. flakybot: issue An issue filed by the Flaky Bot. Should not be added manually. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@flaky-bot
Copy link

flaky-bot bot commented Jan 24, 2023

Note: #873 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.


commit: 830f325
buildURL: Build Status, Sponge
status: failed

Test output
args = (session: "projects/precise-truck-742/instances/google-cloud-1674556583478/databases/test_sessions_1674556697649/sessi...\'bharney@example.com\') RETURNING contact_id, first_name, last_name, email"
params {
}
seqno: 1
request_options {
}
,)
kwargs = {'metadata': [('google-cloud-resource-prefix', 'projects/precise-truck-742/instances/google-cloud-1674556583478/databa...yds5uvmqTsMRtsw'), ('x-goog-api-client', 'gl-python/3.8.13 grpc/1.52.0rc1 gax/2.11.0 gccl/3.27.0')], 'timeout': 3600.0}
result = <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.ABORTED
	details = "Transaction was aborte...95:443 {created_time:"2023-01-24T10:39:00.603487319+00:00", grpc_status:10, grpc_message:"Transaction was aborted."}"
>
prefetch_first = True
@functools.wraps(callable_)
def error_remapped_callable(*args, **kwargs):
    try:
        result = callable_(*args, **kwargs)
        # Auto-fetching the first result causes PubSub client's streaming pull
        # to hang when re-opening the stream, thus we need examine the hacky
        # hidden flag to see if pre-fetching is disabled.
        # https://github.com/googleapis/python-pubsub/issues/93#issuecomment-630762257
        prefetch_first = getattr(callable_, "_prefetch_first_result_", True)
      return _StreamingResponseIterator(
            result, prefetch_first_result=prefetch_first
        )

.nox/system-3-8-database_dialect-postgresql/lib/python3.8/site-packages/google/api_core/grpc_helpers.py:162:


self = <google.api_core.grpc_helpers._StreamingResponseIterator object at 0x7fc8c82495b0>
wrapped = <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.ABORTED
details = "Transaction was aborte...95:443 {created_time:"2023-01-24T10:39:00.603487319+00:00", grpc_status:10, grpc_message:"Transaction was aborted."}"

prefetch_first_result = True

def __init__(self, wrapped, prefetch_first_result=True):
    self._wrapped = wrapped

    # This iterator is used in a retry context, and returned outside after init.
    # gRPC will not throw an exception until the stream is consumed, so we need
    # to retrieve the first result, in order to fail, in order to trigger a retry.
    try:
        if prefetch_first_result:
          self._stored_first_result = next(self._wrapped)

.nox/system-3-8-database_dialect-postgresql/lib/python3.8/site-packages/google/api_core/grpc_helpers.py:88:


self = <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.ABORTED
details = "Transaction was aborte...95:443 {created_time:"2023-01-24T10:39:00.603487319+00:00", grpc_status:10, grpc_message:"Transaction was aborted."}"

def __next__(self):
  return self._next()

.nox/system-3-8-database_dialect-postgresql/lib/python3.8/site-packages/grpc/_channel.py:475:


self = <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.ABORTED
details = "Transaction was aborte...95:443 {created_time:"2023-01-24T10:39:00.603487319+00:00", grpc_status:10, grpc_message:"Transaction was aborted."}"

def _next(self) -> Any:
    with self._state.condition:
        if self._state.code is None:
            event_handler = _event_handler(self._state,
                                           self._response_deserializer)
            self._state.due.add(cygrpc.OperationType.receive_message)
            operating = self._call.operate(
                (cygrpc.ReceiveMessageOperation(_EMPTY_FLAGS),),
                event_handler)
            if not operating:
                self._state.due.remove(cygrpc.OperationType.receive_message)
        elif self._state.code is grpc.StatusCode.OK:
            raise StopIteration()
        else:
            raise self

        def _response_ready():
            return (self._state.response is not None or
                    (cygrpc.OperationType.receive_message
                     not in self._state.due and
                     self._state.code is not None))

        _common.wait(self._state.condition.wait, _response_ready)
        if self._state.response is not None:
            response = self._state.response
            self._state.response = None
            return response
        elif cygrpc.OperationType.receive_message not in self._state.due:
            if self._state.code is grpc.StatusCode.OK:
                raise StopIteration()
            elif self._state.code is not None:
              raise self

E grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
E status = StatusCode.ABORTED
E details = "Transaction was aborted."
E debug_error_string = "UNKNOWN:Error received from peer ipv4:74.125.197.95:443 {created_time:"2023-01-24T10:39:00.603487319+00:00", grpc_status:10, grpc_message:"Transaction was aborted."}"
E >

.nox/system-3-8-database_dialect-postgresql/lib/python3.8/site-packages/grpc/_channel.py:881: _MultiThreadedRendezvous

The above exception was the direct cause of the following exception:

sessions_database = <google.cloud.spanner_v1.database.Database object at 0x7fc8c83f34f0>
sessions_to_delete = [<google.cloud.spanner_v1.session.Session object at 0x7fc8c82466d0>]
database_dialect = <DatabaseDialect.POSTGRESQL: 2>

@_helpers.retry_mabye_conflict
@pytest.mark.skipif(
    _helpers.USE_EMULATOR, reason="Emulator does not support DML Returning."
)
def test_transaction_execute_sql_dml_returning(
    sessions_database, sessions_to_delete, database_dialect
):
    sd = _sample_data

    session = sessions_database.session()
    session.create()
    sessions_to_delete.append(session)

    with session.batch() as batch:
        batch.delete(sd.TABLE, sd.ALL)

    with session.transaction() as transaction:
        for row in sd.ROW_DATA:
            insert_statement = _generate_insert_returning_statement(
                row, database_dialect
            )
            results = transaction.execute_sql(insert_statement)
          returned = results.one()

tests/system/test_session_api.py:808:


google/cloud/spanner_v1/streamed.py:157: in one
answer = self.one_or_none()
google/cloud/spanner_v1/streamed.py:181: in one_or_none
answer = next(iterator)
google/cloud/spanner_v1/streamed.py:145: in iter
self._consume_next()
google/cloud/spanner_v1/streamed.py:117: in _consume_next
response = next(self._response_iterator)
google/cloud/spanner_v1/snapshot.py:83: in _restart_on_unavailable
iterator = method(request=request)
google/cloud/spanner_v1/services/spanner/client.py:1202: in execute_streaming_sql
response = rpc(
.nox/system-3-8-database_dialect-postgresql/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py:113: in call
return wrapped_func(*args, **kwargs)
.nox/system-3-8-database_dialect-postgresql/lib/python3.8/site-packages/google/api_core/timeout.py:120: in func_with_timeout
return func(*args, **kwargs)


args = (session: "projects/precise-truck-742/instances/google-cloud-1674556583478/databases/test_sessions_1674556697649/sessi...'bharney@example.com') RETURNING contact_id, first_name, last_name, email"
params {
}
seqno: 1
request_options {
}
,)
kwargs = {'metadata': [('google-cloud-resource-prefix', 'projects/precise-truck-742/instances/google-cloud-1674556583478/databa...yds5uvmqTsMRtsw'), ('x-goog-api-client', 'gl-python/3.8.13 grpc/1.52.0rc1 gax/2.11.0 gccl/3.27.0')], 'timeout': 3600.0}
result = <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.ABORTED
details = "Transaction was aborte...95:443 {created_time:"2023-01-24T10:39:00.603487319+00:00", grpc_status:10, grpc_message:"Transaction was aborted."}"

prefetch_first = True

@functools.wraps(callable_)
def error_remapped_callable(*args, **kwargs):
    try:
        result = callable_(*args, **kwargs)
        # Auto-fetching the first result causes PubSub client's streaming pull
        # to hang when re-opening the stream, thus we need examine the hacky
        # hidden flag to see if pre-fetching is disabled.
        # https://github.com/googleapis/python-pubsub/issues/93#issuecomment-630762257
        prefetch_first = getattr(callable_, "_prefetch_first_result_", True)
        return _StreamingResponseIterator(
            result, prefetch_first_result=prefetch_first
        )
    except grpc.RpcError as exc:
      raise exceptions.from_grpc_error(exc) from exc

E google.api_core.exceptions.Aborted: 409 Transaction was aborted.

.nox/system-3-8-database_dialect-postgresql/lib/python3.8/site-packages/google/api_core/grpc_helpers.py:166: Aborted

@flaky-bot flaky-bot bot added flakybot: issue An issue filed by the Flaky Bot. Should not be added manually. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Jan 24, 2023
@product-auto-label product-auto-label bot added the api: spanner Issues related to the googleapis/python-spanner API. label Jan 24, 2023
@flaky-bot flaky-bot bot added the flakybot: flaky Tells the Flaky Bot not to close or comment on this issue. label Jan 25, 2023
@flaky-bot
Copy link
Author

flaky-bot bot commented Jan 25, 2023

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.


When run at the same commit (830f325), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/python-spanner API. flakybot: flaky Tells the Flaky Bot not to close or comment on this issue. flakybot: issue An issue filed by the Flaky Bot. Should not be added manually. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

1 participant