Poor termination handling in bidi.BackgroundConsumer
resulting in error logs
#619
Labels
priority: p2
Moderately-important priority. Fix may not be included in next release.
type: bug
Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Overview
caused by poor termination handling logic in the
bidi.BackgroundConsumer
class.has anything to do with the storage write client but instead is related to the
bidi.BackgroundConsumer
class.Environment details
google-api-core
version: 2.17.1Error 1:
bidi.BackgroundConsumer
error log on shutdown using very simple client server implementationcall using the
bidi.BidiRpc
andbidi.BackgroundConsumer
classes which results in an error being logged.Stack trace
Error 2: Debug level exception in
bidi.BackgroundConsumer
on shutdown when using the BigQuery Storage Write APIBigQuery Storage Write API but it shows up differently.
gets wrapped in a
_StreamingResponseIterator
which converts the_MultiThreadedRendezvous
into aGoogleAPICallError
Stack trace
Error 3: Uncaught exception in
bidi.BackgroundConsumer
log when using the BigQuery Storage Write APIoccurs in our production environment (Google AppEngine flexible) 10-100 times per day.
channel seems to be a bit different and the result is an uncaught
StopIteration
exceptionStack trace
Proposed fix
I think just a bit of extra logic in the
bidi.BackgroundConsumer
class to catch and handle theStopIteration
andcanceled exceptions would be sufficient to fix this issue in the second and third examples above. Specifically updating
the
google.api_core.bidi.BackgroundConsumer._thread_main()
implementation with the following block:Moreover catching the _MultiThreadedRendezvous could be done as well although this is admittedly very strange.
A better approach could be to update the
grpc._channel._MultiThreadedRendezvous._next()
implemenation to raiseStopIteration
onStatusCode.CANCELLED
in addition toOK
.So the full, the contents of
bidi.BackgroundConsumer
The text was updated successfully, but these errors were encountered: