Description
Problems with sslproto.py
-
The code is very complex and hard to reason about. At this point it's pretty much unmaintainable, many bugs on bugs.python.org linger unresolved just because we don't have capacity to debug them or review PRs.
-
It's very slow. SSL connections are 3-4x slower than regular connections.
-
It has many yet unresolved problems associated with SSL connection shutdown and cleanup:
-
We don't have enough functional tests for SSL in asyncio. Especially ones that test cancellation & timeouts thoroughly enough.
Requirements for the new implementation
-
Easy to understand code. Ideally, the state machine should be implemented right in the
SSLProtocol(so noSSLPipeabstraction). A good example of a protocol state machine is asyncpg'sCoreProtocol <-> Protocol <-> Connectionmechanism. -
Performance. Ideally, SSL connections shouldn't be more than 10% slower.
-
Tests. Nathaniel once mentioned that Trio's SSL tests are very robust and that he found bugs in CPython's ssl module with them. We maybe want to backport them to asyncio.
-
SSL tunneled through SSL should be naturally supported.
Suggested steps towards the new implementation
-
Try to benchmark the current
asyncio/sslproto.py. Maybe we'll be able to pinpoint the exact bottleneck that makes it so slow; what if there's a bug in ssl module? Most likely there's no single bottleneck, and the reason for poor performance is just its inefficient code. -
Implement a new SSL state machine/protocol in pure Python (I suggest to use new
asyncio.BufferedProtocolfrom the beginning, though.) Get it to the point that it passes asyncio's existing SSL tests. Add more tests. -
Once we have a working pure Python implementation, we can (a) contribute it to Python 3.8, (b) start optimizing it in Cython for uvloop.
cc @fantix