Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-84753: Make inspect.iscoroutinefunction() work with AsyncMock #94050

Merged
merged 2 commits into from
Jun 30, 2022

Conversation

sileht
Copy link
Contributor

@sileht sileht commented Jun 21, 2022

The inspect version was not working with unittest.mock.AsyncMock.

This change moves the asyncio fix for AsyncMock in inspect module and
make asyncio.iscoroutinefunction an alias of inspect.iscoroutinefunction.

@cpython-cla-bot
Copy link

cpython-cla-bot bot commented Jun 21, 2022

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-bot
Copy link

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

@sileht
Copy link
Contributor Author

sileht commented Jun 21, 2022

@vstinner Can you take a look?

@graingert
Copy link
Contributor

this shouldn't be needed AsyncMock already sets

cpython/Lib/unittest/mock.py

Lines 2175 to 2178 in 5fcfdd8

code_mock = NonCallableMock(spec_set=CodeType)
code_mock.co_flags = inspect.CO_COROUTINE
self.__dict__['__code__'] = code_mock

Lib/inspect.py Outdated
@@ -406,12 +406,15 @@ def isgeneratorfunction(obj):
See help(isfunction) for a list of attributes."""
return _has_code_flag(obj, CO_GENERATOR)

# A marker for iscoroutinefunction.
_is_coroutine = object()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this here instead of re-using asyncio.coroutines._is_coroutine?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved it from asyncio to inspect as the code using it is now located in inspect module.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_is_coroutine feels like a property of asyncio, not inspect. I think it should stay where it is.

@sileht
Copy link
Contributor Author

sileht commented Jun 21, 2022

this shouldn't be needed AsyncMock already sets

cpython/Lib/unittest/mock.py

Lines 2175 to 2178 in 5fcfdd8

code_mock = NonCallableMock(spec_set=CodeType)
code_mock.co_flags = inspect.CO_COROUTINE
self.__dict__['__code__'] = code_mock

The test cases I added passed was passing for asyncio version, but was not for the inspect version. So something doesn't work as expected.

@graingert
Copy link
Contributor

looks like this issue is because mock.AsyncMock() doesn't pass inspect.isfunction:

from unittest import mock
import inspect

m = mock.AsyncMock()
with mock.patch("inspect.isfunction", new=lambda x: True):
    assert inspect.iscoroutinefunction(m)

Originally posted by @graingert in #84753 (comment)

@graingert
Copy link
Contributor

graingert commented Jun 21, 2022

Could CallableMixin mess about with __instancecheck__/__subclasscheck__/__class__ to pass isinstance(AsyncMock(), types.FunctionType)?

@sileht
Copy link
Contributor Author

sileht commented Jun 24, 2022

Could CallableMixin mess about with __instancecheck__/__subclasscheck__/__class__ to pass isinstance(AsyncMock(), types.FunctionType)?

I tried but this doesn't seem to work, I suspect this fall in isinstance C optimisation and __instancecheck__ and friends are never called.

@graingert
Copy link
Contributor

I'm relying on inspect.iscoroutinefunction to reveal things that really really are Coroutine Functions, partials or methods and asyncio.iscoroutinefunction to reveal things that are Duck typed as a Coroutine Function

@sileht
Copy link
Contributor Author

sileht commented Jun 24, 2022

I'm relying on inspect.iscoroutinefunction to reveal things that really really are Coroutine Functions, partials or methods and asyncio.iscoroutinefunction to reveal things that are Duck typed as a Coroutine Function

According to the doc, the only difference is asyncio.iscoroutinefunction also works for deprecated generator based coroutine
This legacy coroutine will(have ?) be removed for 3.11, so now both functions should be the same.

@sileht
Copy link
Contributor Author

sileht commented Jun 24, 2022

But, I agree that my solution doesn't work. We should keep the _is_coroutine hack as-is until generated coroutine are removed and found another solution to make isfunction pass for AsyncMock().

@sileht
Copy link
Contributor Author

sileht commented Jun 28, 2022

I found another solution, I think it's cleaner.

@sileht sileht force-pushed the iscoroutinefix branch 3 times, most recently from fe4fc4e to 970f727 Compare June 28, 2022 17:56
Lib/inspect.py Outdated
return False
return bool(f.__code__.co_flags & flag)
if isfunction(f) or _signature_is_functionlike(f):
return bool(f.__code__.co_flags & flag)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you try to preserve current coding style? Return early with False if a condition is false? Pseudo-code:

if not isfunction(f): return False
if not _signature_is_functionlike(f): return False
return bool(f.__code__.co_flags & flag)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@graingert
Copy link
Contributor

I think the PR title should be updated up match the new approach

@sileht sileht changed the title gh-84753: sync iscoroutinefunction of asyncio and inspect module gh-84753: make inspect.iscoroutinefunction() works with AsyncMock Jun 29, 2022
@ambv ambv changed the title gh-84753: make inspect.iscoroutinefunction() works with AsyncMock gh-84753: Make inspect.iscoroutinefunction() work with AsyncMock Jun 30, 2022
@ambv ambv added needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes labels Jun 30, 2022
@ambv ambv merged commit 4261b6b into python:main Jun 30, 2022
@miss-islington
Copy link
Contributor

Thanks @sileht for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 30, 2022
pythonGH-94050)

The inspect version was not working with unittest.mock.AsyncMock.

The fix introduces special-casing of AsyncMock in
`inspect.iscoroutinefunction` equivalent to the one
performed in `asyncio.iscoroutinefunction`.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit 4261b6b)

Co-authored-by: Mehdi ABAAKOUK <sileht@sileht.net>
@bedevere-bot
Copy link

GH-94460 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Jun 30, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 30, 2022
pythonGH-94050)

The inspect version was not working with unittest.mock.AsyncMock.

The fix introduces special-casing of AsyncMock in
`inspect.iscoroutinefunction` equivalent to the one
performed in `asyncio.iscoroutinefunction`.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit 4261b6b)

Co-authored-by: Mehdi ABAAKOUK <sileht@sileht.net>
@bedevere-bot bedevere-bot removed the needs backport to 3.10 only security fixes label Jun 30, 2022
@bedevere-bot
Copy link

GH-94461 is a backport of this pull request to the 3.10 branch.

@sileht
Copy link
Contributor Author

sileht commented Jun 30, 2022

Thanks all! That was fast 🚀

ambv pushed a commit that referenced this pull request Jun 30, 2022
…94050) (GH-94461)

The inspect version was not working with unittest.mock.AsyncMock.

The fix introduces special-casing of AsyncMock in
`inspect.iscoroutinefunction` equivalent to the one
performed in `asyncio.iscoroutinefunction`.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit 4261b6b)

Co-authored-by: Mehdi ABAAKOUK <sileht@sileht.net>
ambv pushed a commit that referenced this pull request Jun 30, 2022
…94050) (GH-94460)

The inspect version was not working with unittest.mock.AsyncMock.

The fix introduces special-casing of AsyncMock in
`inspect.iscoroutinefunction` equivalent to the one
performed in `asyncio.iscoroutinefunction`.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit 4261b6b)

Co-authored-by: Mehdi ABAAKOUK <sileht@sileht.net>
ambv added a commit to ambv/cpython that referenced this pull request Jul 5, 2022
The wording used in pythonGH-94050 suggests narrower scope than what was actually
implemented. This commit clarifies the full impact of pythonGH-94050 in the change
log.
@tirkarthi
Copy link
Member

This change seems to have caused a regression reported in #96127

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants