asyncio: potential leak of TLS connections

[romuald]

Bug report

Synopsis

Forgetting to close TLS connections manually with asyncio.open_connection() will lead to a leak of TCP connection when the writer/reader get out of scope

Note: the reference is properly released when the remote side closes the connection

This seems to be counter intuitive relative to other python APIs where the connection is closed when the handle goes out of scope

Details

• open a TLS connection with asyncio.open_connection(..., ssl=True)
• do some read/writes
• exit function, so handlers get out of scope (and possibly gc collected). This may be due to an exception for example
• do not call writer.close()
• the connection is now "unreachable" from a user point of view
• however the TCP connection is kept alive

When trying to debug this issue I found out that a _SSLProtocolTransport instance is kept in memory, probably linked to the eventloop

Example script

import os
import asyncio
import gc
import signal

HOST = "google.fr"  # will keep the connection alive for a few minutes at least


async def query():
    reader, writer = await asyncio.open_connection(HOST, 443, ssl=True)

    # No connection: close, remote side will keep the connection open
    writer.write(f"GET / HTTP/1.1\r\nHost: {HOST}\r\n\r\n".encode())
    await writer.drain()

    # only read the first header line
    try:
        return (await reader.readline()).decode()
    finally:
        # closing the writer will properly finalize the connection
        # writer.close()
        pass

    # reader and writer are now unreachable


async def amain():
    await query()

    # The _SSLProtocolTransport object is kept in memory and the
    # connection won't be released until the remote side closes the connection
    for _ in range(200):
        # Just be sure everything is freed, just in case
        gc.collect()

        await asyncio.sleep(1)


def main():
    print(f"PID {os.getpid()}")
    task = asyncio.ensure_future(amain())

    loop = asyncio.get_event_loop()
    loop.add_signal_handler(signal.SIGTERM, task.cancel)
    loop.add_signal_handler(signal.SIGINT, task.cancel)
    loop.run_until_complete(task)


if __name__ == "__main__":
    main()

Your environment

• CPython versions tested on:
  • 3.11.4
  • 3.10.11
• Operating system and architecture: Debian Linux 5.19 x86_64

Linked PRs

• GH-106684: raise ResourceWarning when StreamWriter is not closed #107650
• [3.12] GH-106684: Close asyncio.StreamWriter when asyncio.StreamWriter is not closed by application (GH-107650) #107656
• [3.11] GH-106684: raise ResourceWarning when asyncio.StreamWriter is not closed (GH-107650) #107657
• [3.11] [3.12] GH-106684: Close asyncio.StreamWriter when asyncio.StreamWriter is not closed by application (GH-107650) (GH-107656) #107836