Fixes for allowing making a mixture of unmocked and mocked HTTPS requests using aiohttp #213
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| Entry.single_register(Entry.GET, self.target_url, status=404) | ||
|
|
||
| async with aiohttp.ClientSession(timeout=self.timeout) as session: | ||
| async with session.get(self.target_url, ssl=False) as get_response: |
Check failure
Code scanning / SonarCloud
Server certificates should be verified during SSL/TLS connections
| @pytest.mark.skipif('os.getenv("SKIP_TRUE_HTTP", False)') | ||
| async def test_mocked_https_request_after_unmocked_https_request(self): | ||
| async with aiohttp.ClientSession(timeout=self.timeout) as session: | ||
| response = await session.get(self.target_url + "real", ssl=False) |
Check failure
Code scanning / SonarCloud
Server certificates should be verified during SSL/TLS connections
| async with Mocketizer(None): | ||
| Entry.single_register(Entry.GET, self.target_url + "mocked", status=404) | ||
| async with aiohttp.ClientSession(timeout=self.timeout) as session: | ||
| response = await session.get(self.target_url + "mocked", ssl=False) |
Check failure
Code scanning / SonarCloud
Server certificates should be verified during SSL/TLS connections
| @pytest.mark.skipif('os.getenv("SKIP_TRUE_HTTP", False)') | ||
| async def test_mocked_https_request_after_unmocked_https_request(self): | ||
| async with aiohttp.ClientSession(timeout=self.timeout) as session: | ||
| response = await session.get(self.target_url + "real", ssl=False) |
Check failure
Code scanning / SonarCloud
Server certificates should be verified during SSL/TLS connections
| async with Mocketizer(None): | ||
| Entry.single_register(Entry.GET, self.target_url + "mocked", status=404) | ||
| async with aiohttp.ClientSession(timeout=self.timeout) as session: | ||
| response = await session.get(self.target_url + "mocked", ssl=False) |
Check failure
Code scanning / SonarCloud
Server certificates should be verified during SSL/TLS connections
…after SSL handshake Python 3.11's asyncio.sslproto implementation attempts to read from the SSL object right after completing handshake. python/cpython@13c10bf#diff-0ae38bdc337cc724282d20111dc780b8a9c07385c80476cf304d5b3c9ec306ecR603
Python 3.11's asyncio.sslproto implementation may use a memoryview for the buffer python/cpython@13c10bf#diff-0ae38bdc337cc724282d20111dc780b8a9c07385c80476cf304d5b3c9ec306ecR431-R432
aiohttp calls 'set_default_verify_paths' on the SSL context object https://github.com/aio-libs/aiohttp/blob/v3.9.1/aiohttp/connector.py#L935
This fails like so: > async def read(self) -> _T: > if not self._buffer and not self._eof: > assert not self._waiter > self._waiter = self._loop.create_future() > try: > > await self._waiter > E aiohttp.client_exceptions.ClientOSError: [Errno 1] [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2580) > > .devenv/state/venv/lib/python3.11/site-packages/aiohttp/streams.py:622: ClientOSError
ento
force-pushed
the
aiohttp-cache-clear
branch
from
88c2cc4 to
918cf1c
Compare
|
|
Hi @ento, |
|
Sure, opened a few issues: #215: The primary issue I wanted to fix #209 corresponds to the middle two bullet points (Can't make mocked HTTPS requests using aiohttp and Python 3.11) If it's this repo's general process to require an issue when opening a PR, it'll be useful to have a 'CONTRIBUTING' file or a section in the README that instructs contributors about the process - will save one round of back-and-forth for new contributors and the maintainer alike :) |
I agree, but please note Mocket is still, and probably it'll ever be, a very small project. |
|
Hi @ento, I merged all your commits, with a very small change to get rid of an unnecessary |
|
Re: issue vs PR - that's fair. Thanks for the new release! Re: the |
|
Hi @ento, I am sorry if I haven't made any progress with your feature proposal related to the strict mode. |
|
@mindflayer No worries, I'm not exactly blocked by the feature proposal getting taken up (I can always use a temporary fork). Your well-being comes first! |
2 participants
This PR attempts to fix a few issues:
In a project I'm working with, a mocketized test would fail if it ran after a test that makes a real HTTPS request. Both use aiohttp. Clearing aiohttp's internal cache that holds an SSLContext object fixed the issue.
I encountered more issues while trying to write a PR for the above:
Mocket.r_fdfromMocketSocket.filenoif it's already set and closing the pipe's file descriptors when resetting MocketSSLWantReadErrorgets raised and get retried later. With Mocket, it returns an empty string in that case, which causes the connection to get shut down. Tracing function calls using hunter was helpful in determining the cause here.SSLWantReadErrorwhen SSL handshake occurred and no bytes have been sent back yet. This works with my project's test suite, but admittedly I'm not sure if there's a case where this would cause an infinite loop waiting for a response or some other failure. But I guess that'd correspond to timeouts, which are something that can happen in real-life scenarios too.MocketSocket.recv_intomay receive a memorybuffer objectwriteattributessl=Falsefailed because it calls a method on SSLContext object that wasn't mocked by MocketFakeSSLContext.DUMMY_METHODS