tests - add test for ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object …

[commonism]

What do these changes do?

Add a test for
ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at …
when using aiohttp.ClientSession with TLS

Are there changes in behavior for the user?

No

Related issue number

#4703
#4713

[webknjaz]

@commonism Git setup on your computer looks broken: none of the submitted commits list your GitHub account as an author. You should probably configure Git locally or verify the email that is embedded in those commits in your GitHub account.

[webknjaz]

Thanks for the effort! Unfortunately, this doesn't look ready, nor is compliant with our test writing practices.

Also, there's great prior art on the usage of pytest fixtures including proper TLS setup — https://github.com/aio-libs/aiohttp/pull/4713/files — why haven't you start there? It also uses proper loop isolation, and pytest-based warnings tracking.

Besides, adding a custom server/client implementations doesn't seem related to the problem you're attempting to reproduce — it only makes the test module messy and distracts from what's actually being tested.

[asvetlov]

@commonism I respect your striving to point on the very annoying issue.
It will be never fixed on aiohttp 3.x branch, sorry.
aiohttp 4.0 has partially done work for getting rid of the warning. The missing part is #5102

P.S.
aiohttp uses pytest with pytest-aiohttp plugin, not the standard unittest.
This is the deliberate choice. We don't accept tests that are unittest based.

P.P.S.
I'm aware of unittest.IsolatedAsyncioTestCase because I had added it to the CPython standard library.
It exists because CPython itself cannot depend on pytest project for running self-testing.
For third-party libs I always prefer pytest-style.

[commonism]

I addressed the following issues

• pytest, not a unittest
• no logging/print
• no tracemalloc
• reduce time required to 5s (was 30 seconds actually)
• using the internal trustme CA

[commonism]

addressing the issues.

[webknjaz]

@commonism we also need the CI to actually execute this test. Red linting job prevents that from happening.

[webknjaz]

@commonism P.S. your local Git setup is still broken

[commonism]

@commonism we also need the CI to actually execute this test. Red linting job prevents that from happening.

The Red Linter is gone, now the tests fail after 2 minutes - timeout?

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ commonism
❌ common

---

common seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[commonism]

Hmm, there's a TimeoutError in 3.12 though... Not sure what's happening there. It also passes on pypy.

Actually the timer got canceld?

self = <aiohttp.helpers.TimerContext object at 0x7ff31767dac0>
exc_type = <class 'asyncio.exceptions.CancelledError'>
exc_val = CancelledError(), exc_tb = <traceback object at 0x7ff3192e1d00>

    def __exit__(
        self,
        exc_type: Optional[Type[BaseException]],
        exc_val: Optional[BaseException],
        exc_tb: Optional[TracebackType],
    ) -> Optional[bool]:
        if self._tasks:
            self._tasks.pop()  # type: ignore[unused-awaitable]
    
        if exc_type is asyncio.CancelledError and self._cancelled:
>           raise asyncio.TimeoutError from None
E           TimeoutError

exc_tb     = <traceback object at 0x7ff3192e1d00>
exc_type   = <class 'asyncio.exceptions.CancelledError'>
exc_val    = CancelledError()
self       = <aiohttp.helpers.TimerContext object at 0x7ff31767dac0>

aiohttp/helpers.py:723: TimeoutError

https://github.com/aio-libs/aiohttp/actions/runs/5978392347/job/16220898181

[Dreamsorcerer]

Actually the timer got canceld?

Hmm, we switched to asyncio.timeout() recently. So, I've merged master and will see if that helps.

[commonism]

I guess it's the 300s ClientTimeout

    real_timeout = ClientTimeout(total=300, connect=None, sock_read=None, sock_connect=None, ceil_threshold=5)

300.26s call tests/test_ssl_transport_close.py::test_unclosed_transport_asyncio_sslproto_SSLProtocolTransport[pyloop]

increasing the sleep to something larger to have to server restart in time?

[Dreamsorcerer]

I'm not seeing how the sleep relates to it, but we can try it if you think that's the problem.

[commonism]

In 3.12 it hangs here - you do not get

DEBUG aiohttp.server:web_protocol.py:434 Ignored premature client disconnection

as you get in 3.11

[Dreamsorcerer]

I don't see anything suspicious in there, unless the asyncio server is not closing:
https://github.com/commonism/aiohttp/blob/ebe2a48c282f2484ee1766af5c4d52ff3431b43a/aiohttp/web_runner.py#L83

Everything else looks like it has a 60s timeout, so would fail before the 5 min timeout we're seeing.

[Dreamsorcerer]

If it is the asyncio server not closing, then we should get a cpython bug filed for the regression.

[Dreamsorcerer]

This is actually working now. But, I don't know if we want to merge such a complex test after realising that these warnings were already visible in many tests (just ignored in the pytest config), so we already have coverage of this.