ConnectionResetError: Cannot write to closing transport

[soyasoya5]

Python version: Python 3.8.0

Sample Code

import aiohttp
import asyncio
from aiohttp_socks import ProxyConnector

class Example:
    async def create_session(self):
        connector = ProxyConnector.from_url('socks4://13.0.0.2:1080')
        self.s = aiohttp.ClientSession(connector=connector)

    async def close_session(self):
        await self.s.close()

    async def send_request(self):
        async with self.s.get('https://google.com') as r:
            print(r.status)

async def task_helper(example, s_time):
    await asyncio.sleep(s_time)
    await example.send_request()

async def main():
    example = Example()
    try:
        await example.create_session()
        tasks = [asyncio.create_task(task_helper(example, i)) for i in range(0,15,3)]
        await asyncio.gather(*tasks)
    finally:
        await example.close_session()

if __name__ == '__main__':
    asyncio.run(main())

Error

An open stream object is being garbage collected; call "stream.close()" explicitly.
Traceback (most recent call last):
  File "/home/user/pyvenv/aiohttp/lib/python3.8/site-packages/aiohttp/client.py", line 542, in _request
    resp = await req.send(conn)
  File "/home/user/pyvenv/aiohttp/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 668, in send
    await writer.write_headers(status_line, self.headers)
  File "/home/user/pyvenv/aiohttp/lib/python3.8/site-packages/aiohttp/http_writer.py", line 119, in write_headers
    self._write(buf)
  File "/home/user/pyvenv/aiohttp/lib/python3.8/site-packages/aiohttp/http_writer.py", line 67, in _write
    raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "test.py", line 31, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete
    return future.result()
  File "test.py", line 26, in main
    await asyncio.gather(*tasks)
  File "test.py", line 19, in task_helper
    await example.send_request()
  File "test.py", line 14, in send_request
    async with self.s.get('https://google.com') as r:
  File "/home/user/pyvenv/aiohttp/lib/python3.8/site-packages/aiohttp/client.py", line 1117, in __aenter__
    self._resp = await self._coro
  File "/home/user/pyvenv/aiohttp/lib/python3.8/site-packages/aiohttp/client.py", line 554, in _request
    raise ClientOSError(*exc.args) from exc
aiohttp.client_exceptions.ClientOSError: Cannot write to closing transport

I believe the socks proxy works, as the same async code works using httpx, and synchronous code works using requests-socks

[romis2012]

Your code works perfectly with alive socks. Try using another proxy

[romis2012]

Another possible reason is that google added your socks to the blacklist. Try another target resource too

[soyasoya5]

I'm getting the error above no matter what proxy I use, the proxies are verified to be working by using curl --proxy
I'm running Ubuntu 18.04.1 LTS

Here's my pip freeze output if that helps

aiohttp==3.7.4.post0
aiohttp-retry==2.4.6
aiohttp-socks==0.6.0
anyio==3.3.2
async-timeout==3.0.1
attrs==21.2.0
certifi==2021.5.30
chardet==4.0.0
charset-normalizer==2.0.6
h11==0.12.0
httpcore==0.13.7
httpx==0.19.0
httpx-socks==0.4.1
idna==3.2
multidict==5.1.0
pkg-resources==0.0.0
PySocks==1.7.1
python-dotenv==0.19.0
python-socks==1.2.4
redis==3.5.3
requests==2.26.0
rfc3986==1.5.0
sniffio==1.2.0
typing-extensions==3.10.0.2
urllib3==1.26.7
uvloop==0.16.0
yarl==1.6.3

[soyasoya5]

Logs from proxy server when running the script above

danted[198997]: info: pass(1): tcp/accept [: 13.0.0.x.57038 13.0.0.2.1080
danted[195200]: info: pass(1): tcp/accept ]: 6660 -> 13.0.0.x.57038 13.0.0.2.1080 -> 597: local client closed.  Session duration: 0s

[romis2012]

I tested your code with different types of free proxies from here http://free-proxy.cz. This works perfectly for me. Please provide the simplest code sample that fails with any proxy.

[soyasoya5]

I'm using the exact same example code, the same error is returned from any proxy. Any ideas on how to proceed from here?

[romis2012]

I have no idea. Your code works correctly for me with any proxy.

[iosakurov]

У меня такая же проблема, прокси рабочие, но вот конкретно через эту библиотеку не работают. Хз куда копать

[khoben]

This error occurs when accessing some sites via proxyconnector (http, socks5), but if specify http proxy directly in the request all is OK.

[Rongronggg9]

@khoben Are you using CPython 3.11.5? It seems to be a regression in 3.11.5.

import asyncio
import aiohttp
from aiohttp_socks import ProxyConnector


async def fetch(url):
    connector = ProxyConnector.from_url('socks5://127.0.0.1:1080')

    async with aiohttp.ClientSession(connector=connector) as session:
        async with session.get(url) as response:
            return f'{response.status} {response.reason}'


if __name__ == '__main__':
    print(asyncio.run(fetch('http://example.com')))

$ docker run --rm -v $PWD/test.py:/test.py --network host python:3.11.4-slim sh -c 'pip install -qq aiohttp_socks; python test.py'
200 OK
$ docker run --rm -v $PWD/test.py:/test.py --network host python:3.11.5-slim sh -c 'pip install -qq aiohttp_socks; python test.py'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 558, in _request
    resp = await req.send(conn)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 670, in send
    await writer.write_headers(status_line, self.headers)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/http_writer.py", line 130, in write_headers
    self._write(buf)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/http_writer.py", line 75, in _write
    raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "//test.py", line 15, in <module>
    print(asyncio.run(fetch('http://example.com')))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "//test.py", line 10, in fetch
    async with session.get(url) as response:
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 572, in _request
    raise ClientOSError(*exc.args) from exc
aiohttp.client_exceptions.ClientOSError: Cannot write to closing transport

Not sure if python/cpython#107913 has fixed the regression. I've not done any tests yet.

@romis2012 Could you reopen the issue until the next CPython release? So that others could find it easily. Or I can open a new one if you'd prefer.

[Rongronggg9]

Not sure if python/cpython#107913 has fixed the regression. I've not done any tests yet.

Until the latest commit on the 3.11 branch (python/cpython@79f7a4c), the regression persists.

Before we can report the issue to CPython, we need to figure out a minimal reproducer, @romis2012, could you help?

[romis2012]

Before we can report the issue to CPython, we need to figure out a minimal reproducer, @romis2012, could you help?

I can't reproduce this issue, as I wrote above

[khoben]

@khoben Are you using CPython 3.11.5? It seems to be a regression in 3.11.5.

Yes, it looks like it broke after August 26 when the docker container with the base image python:3.11-slim-bullseye was rebuilt and updated to 3.11.5.