Skip to content

Conversation

@canonrock16
Copy link
Contributor

Overview

  • prevent repeatedly wrapping aiomysql pooled connections with AsyncConnectionWrapper
  • avoid building deep ObjectProxy chains that led to RecursionError: maximum recursion depth exceeded
  • add a regression unit test (tests/agent_unittests/test_database_aiomysql.py) that simulates a pool returning the same connection twice

Related Github Issue

None.
Related forum thread: https://support.newrelic.com/s/hubtopic/aAXPh000000EHIjOAO/recursionerror-when-new-relic-python-agent-instruments-aiomysql-still-happening

Testing

  • tox -c tox.ini -e python-agent_unittests-py312-without_extensions
    All tests passed including the new test I added

@CLAassistant
Copy link

CLAassistant commented Oct 23, 2025

CLA assistant check
All committers have signed the CLA.

@TimPansino
Copy link
Contributor

I'll handle moving the tests for this to the right place and making sure they're working, thanks for finding this though.

Really curious how Pool._acquire() is maintaining the wrappers such that they're preserved for future calls. I'll have to read the codebase again and look into it.

@TimPansino TimPansino self-assigned this Oct 23, 2025
@canonrock16
Copy link
Contributor Author

Reproduction details

Agent 10.17.0 / aiomysql 0.2.0 / Python 3.12
Applied load against the service with hey; during the run the service began returning 500s and CloudWatch logs filled with RecursionError: maximum recursion depth exceeded
Full write-up: https://support.newrelic.com/s/hubtopic/aAXPh000000EHIjOAO/recursionerror-when-new-relic-python-agent-instruments-aiomysql-still-happening

Sample log :

2025-10-22T08:52:10.099Z RecursionError: maximum recursion depth exceeded
2025-10-22T08:52:10.184Z ERROR:root:unexpected error
File "<internal_path>/server.py", line 105, in wrap_error
return await func(*args, **kwargs)
...
File "<internal_path>/newrelic/hooks/database_aiomysql.py", line 79, in _wrap_pool__acquire
connection = await wrapped(*args, **kwargs)
File "<internal_path>/aiomysql/pool.py", line 171, in _fill_free_pool
self._loop.time() - conn.last_usage > self._recycle)
File "<internal_path>/newrelic/common/object_wrapper.py", line 70, in getattr
return super().getattr(name)
[previous line repeated 322 more times]
RecursionError: maximum recursion depth exceeded

After this patch, the same load no longer produces RecursionErrors and the service stays healthy.

@canonrock16
Copy link
Contributor Author

I'll handle moving the tests for this to the right place and making sure they're working, thanks for finding this though.

Really curious how Pool._acquire() is maintaining the wrappers such that they're preserved for future calls. I'll have to read the codebase again and look into it.

For reference, aiomysql’s pool implementation reuses the same connection object pulled from the free queue:

_acquire() pops a connection from self._free, and _fill_free_pool() rotates existing entries unless they’re closed or expired. As a result, the same instance that was returned to the pool is handed back on the next acquire(), so the agent’s wrapper stays attached unless we detect it and avoid wrapping again.

@canonrock16 canonrock16 force-pushed the fix/aiomysql-recursion branch from 5f6234f to ee8b864 Compare October 24, 2025 00:12
@canonrock16 canonrock16 marked this pull request as ready for review October 24, 2025 00:42
@canonrock16 canonrock16 requested a review from a team as a code owner October 24, 2025 00:42
@TimPansino TimPansino enabled auto-merge (squash) October 24, 2025 19:33
@TimPansino TimPansino merged commit 996ee66 into newrelic:main Oct 24, 2025
59 checks passed
@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.76%. Comparing base (fcaeb10) to head (5887526).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1553      +/-   ##
==========================================
- Coverage   81.76%   81.76%   -0.01%     
==========================================
  Files         207      207              
  Lines       23946    23948       +2     
  Branches     3796     3797       +1     
==========================================
+ Hits        19580    19581       +1     
  Misses       3100     3100              
- Partials     1266     1267       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants