Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix httpx client creating a new ssl context with each client (memory leak) #90191

Merged
merged 2 commits into from
Mar 24, 2023

Conversation

bdraco
Copy link
Member

@bdraco bdraco commented Mar 23, 2023

Proposed change

While working on #83524 it was discovered that each new httpx client creates a new ssl context even when ssl verify is disabled.

https://github.com/encode/httpx/blob/f1157dbc4102ac8e227a0a0bb12a877f592eff58/httpx/_transports/default.py#L261

If an ssl context is passed in creating a new one is avoided here

https://github.com/encode/httpx/blob/f1157dbc4102ac8e227a0a0bb12a877f592eff58/httpx/_config.py#L110

This change makes httpx ssl no-verify behavior match aiohttp ssl no-verify behavior

https://github.com/aio-libs/aiohttp/blob/6da04694fd87a39af9c3856048c9ff23ca815f88/aiohttp/connector.py#L892

aiohttp solved this by wrapping the code that generates the ssl context in an lru_cache

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

While working on #83524
it was discovered that each new httpx client creates a new ssl context

https://github.com/encode/httpx/blob/f1157dbc4102ac8e227a0a0bb12a877f592eff58/httpx/_transports/default.py#L261

If an ssl context is passed in creating a new one is avoided here

https://github.com/encode/httpx/blob/f1157dbc4102ac8e227a0a0bb12a877f592eff58/httpx/_config.py#L110

This change makes httpx ssl no-verify behavior match aiohttp ssl no-verify
behavior

https://github.com/aio-libs/aiohttp/blob/6da04694fd87a39af9c3856048c9ff23ca815f88/aiohttp/connector.py#L892

aiohttp solved this by wrapping the code that generates the ssl context
in an lru_cache
bdraco added a commit to bdraco/python-onvif-zeep-async that referenced this pull request Mar 23, 2023
@bdraco bdraco marked this pull request as ready for review March 23, 2023 20:00
@bdraco bdraco requested review from epenet and a team as code owners March 23, 2023 20:00
@MartinHjelmare MartinHjelmare merged commit 1f2268a into dev Mar 24, 2023
@MartinHjelmare MartinHjelmare deleted the httpx_no_ssl_leak_fix branch March 24, 2023 07:40
@bdraco
Copy link
Member Author

bdraco commented Mar 24, 2023

Thanks

hunterjm pushed a commit to openvideolibs/python-onvif-zeep-async that referenced this pull request Mar 24, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Mar 25, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants