0.22.0 - object of type 'asyncpg.pgproto.pgproto.WriteBuffer' has no len()

[rafalstapinski]

• asyncpg version: 0.22.0
• PostgreSQL version: 11
• Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
  the issue with a local PostgreSQL install?:
• Python version: CPython 3.8.1
• Platform: Ubuntu 18.04
• Do you use pgbouncer?: no
• Did you install asyncpg with pip?: pipenv
• If you built asyncpg locally, which version of Cython did you use?: n/a
• Can the issue be reproduced under both asyncio and
  uvloop?: Running under uvloop

After having upgraded our underlying asyncpg version from 0.21.0 to 0.22.0 two days ago, every ~12 hours our http service has been running into an issue at which it can no longer connect to the database without restarting. The issue being reported is

StopIteration: null
  File "asyncpg/protocol/coreproto.pyx", line 945, in asyncpg.protocol.protocol.CoreProtocol._bind_execute_many_more
    buf = <WriteBuffer>next(self._execute_iter)

TypeError: object of type 'asyncpg.pgproto.pgproto.WriteBuffer' has no len()
  File "asyncpg/protocol/protocol.pyx", line 909, in asyncpg.protocol.protocol.BaseProtocol._writelines
    self.transport.writelines(buffers)
  File "uvloop/sslproto.pyx", line 163, in uvloop.loop._SSLProtocolTransport.writelines
    self._ssl_protocol._write_appdata(list_of_data, self.context.copy())
  File "uvloop/sslproto.pyx", line 652, in uvloop.loop.SSLProtocol._write_appdata
    self._write_buffer_size += len(data)

For context, we use tortoise-orm to interface with asyncpg.

[elprans]

@fantix Seems like bad interation of the new executemany code with TLS codepath in uvloop.

[khvn26]

https://github.com/MagicStack/py-pgproto/blob/126bcd24bd3c59d581613dae026e2721efbedf16/buffer.pxd#L28 Shouldn't this be __len__?

[ale-dd]

I am hitting this issue as well, on 0.22.0 and using the new optimized executemany.

[fantix]

https://github.com/MagicStack/py-pgproto/blob/126bcd24bd3c59d581613dae026e2721efbedf16/buffer.pxd#L28 Shouldn't this be __len__?

That's indeed the issue I think. Though I'm not sure if we want to pass over the WriterBuffer directly to the transport - wrapping into a memoryview probably makes more sense.

[Gyllsdorff]

@fantix

Do you know when the fix for this will be release? Is this a warning log or does it mean that sql commands are failing?