Skip to content

gh-75988: Fix issues with autospec ignoring wrapped object #115223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Mar 8, 2024
Merged

gh-75988: Fix issues with autospec ignoring wrapped object #115223

merged 21 commits into from
Mar 8, 2024

Conversation

infohash
Copy link
Contributor

@infohash infohash commented Feb 9, 2024

Fix for:

Description

wraps is an argument defined in unittest.mock.Mock. Here's how official documentation defines it:

wraps: Item for the mock object to wrap. If wraps is not None then calling the Mock will pass the call through to the wrapped object (returning the real result). Attribute access on the mock will return a Mock object that wraps the corresponding attribute of the wrapped object (so attempting to access an attribute that doesn’t exist will raise an AttributeError).

While this argument works as intended, it is not respected by create_autospec . This is the signature of create_autospec:

cpython/Lib/unittest/mock.py

Lines 2704 to 2705 in b104360

def create_autospec(spec, spec_set=False, instance=False, _parent=None,
_name=None, *, unsafe=False, **kwargs):

kwargs takes wraps as a keyword argument.

Tests

Below are the tests that are currently failing because of the issue and will pass after merging the fix.

from unittest import mock

import pytest


class MyClass:
    @staticmethod
    def my_func():
        raise ValueError


def test_my_func_with_autospec():
    my_func = mock.create_autospec(spec=MyClass.my_func, wraps=MyClass.my_func)
    with pytest.raises(ValueError):
        assert my_func()
def wrapped_func(value):
    raise ValueError(value)


@pytest.mark.xfail
def test_wrapped_func():
    with mock.patch('__main__.wrapped_func', autospec=True, wraps=wrapped_func), pytest.raises(
            ValueError):
        wrapped_func('testing')

Details

There are 3 issues that are making autospec to ignore wraps.

Issue 1

When autospec is used on a function/method, autospec replaces the original function/method with its own hardcoded mock function and mock attributes. This can be seen here 👇 and it is intended by design.

cpython/Lib/unittest/mock.py

Lines 2775 to 2781 in 17689e3

if isinstance(spec, FunctionTypes):
# should only happen at the top level because we don't
# recurse for functions
if is_async_func:
mock = _set_async_signature(mock, spec)
else:
mock = _set_signature(mock, spec)

This if condition becomes true when you pass a function or a method to spec argument. This is what FunctionTypes is 👇:

cpython/Lib/unittest/mock.py

Lines 2886 to 2891 in 17689e3

FunctionTypes = (
# python function
type(create_autospec),
# instance method
type(ANY.__eq__),
)

On condition true, _set_signature or _setup_async_mock will return a hardcoded mock function 👇:

src = """def %s(*args, **kwargs):
_checksig_(*args, **kwargs)
return mock(*args, **kwargs)""" % name
exec (src, context)
funcopy = context[name]
_setup_func(funcopy, mock, sig)
return funcopy

Now the question is how this leads to ignoring wraps? See the above line 206 👆 _setup_func, it is patching the return value of the mock function 👇:

funcopy.return_value = mock.return_value

mock.return_value is a property 👇:

return_value = property(__get_return_value, __set_return_value,
__return_value_doc)

So, accessing its value triggers the getter descriptor 👇:

def __get_return_value(self):
ret = self._mock_return_value
if self._mock_delegate is not None:
ret = self._mock_delegate.return_value
if ret is DEFAULT:
ret = self._get_child_mock(
_new_parent=self, _new_name='()'
)
self.return_value = ret
return ret

At line 580 👆, it reassigns a new value to mock.return_value which triggers the setter descriptor 👇:

def __set_return_value(self, value):
if self._mock_delegate is not None:
self._mock_delegate.return_value = value
else:
self._mock_return_value = value
_check_and_set_parent(self, value, None, '()')

At line 588 👆 self._mock_return_value is assigned a value replacing the default value sentinel.DEFAULT. self._mock_return_value will come to the picture soon.

Now the original function has been replaced with the mock and its return value is patched, when it is called, these are the calls made 👇:

cpython/Lib/unittest/mock.py

Lines 1159 to 1164 in 17689e3

def __call__(self, /, *args, **kwargs):
# can't use self in-case a function / method we are mocking uses self
# in the signature
self._mock_check_sig(*args, **kwargs)
self._increment_mock_call(*args, **kwargs)
return self._mock_call(*args, **kwargs)

In the above 👆 return statement self._mock_call(*args, **kwargs) executes 👇:

cpython/Lib/unittest/mock.py

Lines 1167 to 1168 in 17689e3

def _mock_call(self, /, *args, **kwargs):
return self._execute_mock_call(*args, **kwargs)

Inside self._execute_mock_call(*args, **kwargs) 👇:

cpython/Lib/unittest/mock.py

Lines 1234 to 1240 in 17689e3

if self._mock_return_value is not DEFAULT:
return self.return_value
if self._mock_wraps is not None:
return self._mock_wraps(*args, **kwargs)
return self.return_value

self._mock_return_value is no longer sentinel.DEFAULT. Because of that, the line 1234 👆 becomes true and you are returned mocked return value ignoring your wraps which is just below this condition.

Solution

Instead of this:

funcopy.return_value = mock.return_value

the default return value should be mock._mock_return_value

- funcopy.return_value = mock.return_value 
+ funcopy.return_value = mock._mock_return_value

Issue 2

cpython/Lib/unittest/mock.py

Lines 2788 to 2790 in 17689e3

if is_type and not instance and 'return_value' not in kwargs:
mock.return_value = create_autospec(spec, spec_set, instance=True,
_name='()', _parent=mock)

If wraps is provided by the user, it should be passed here so that the instance of the mock can also wrap those child attributes that are also the attribute on the parent (like methods).

Solution

+ wrapped = kwargs.get('wraps')

if is_type and not instance and 'return_value' not in kwargs:
        mock.return_value = create_autospec(spec, spec_set, instance=True,
                                            _name='()', _parent=mock,
+                                           wraps=wrapped)

Issue 3

Inside this for loop:

for entry in dir(spec):

kwargs = {'spec': original}

Child attributes are not being wrapped by the mock.

Solution

It should be like this:

+ wrapped = kwargs.get('wraps')

for entry in dir(spec):

    ...

    kwargs = {'spec': original}
+   if wrapped and hasattr(wrapped, entry):
+       kwargs.update(wraps=original)
    if spec_set:
        kwargs = {'spec_set': original}

Tests for Issue 2 & Issue 3

from unittest.mock import create_autospec
class Result:

    @staticmethod
    def get_result():
        return "real result"

Without patch

>>> result_mock = mock.Mock(spec=Result, wraps=Result)
>>> result_mock.get_result() == "real result"
True

>>> result_mock = mock.create_autospec(spec=Result, wraps=Result)
>>> result_mock.get_result.return_value = mock.DEFAULT
>>> result_mock.get_result() == "real result"
False

With patch

>>> result_mock = mock.Mock(spec=Result, wraps=Result)
>>> result_mock.get_result() == "real result"
True

>>> result_mock = mock.create_autospec(spec=Result, wraps=Result)
>>> result_mock.get_result.return_value = mock.DEFAULT
>>> result_mock.get_result() == "real result"
True

This fix ensures that create_autospec must respect this argument when creating mock objects.

@infohash infohash requested a review from cjw296 as a code owner February 9, 2024 17:28
@ghost
Copy link

ghost commented Feb 9, 2024

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Feb 9, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

1 similar comment
@bedevere-app
Copy link

bedevere-app bot commented Feb 9, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Feb 9, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Feb 9, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Feb 10, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@cjw296
Copy link
Contributor

cjw296 commented Feb 11, 2024

@infohash - your description is a bit terse. Is wraps a new parameter? Where's the documentation for it in either new or existing form? Can you provide some narrative rather than just a couple of test examples?

I'm probably not going to have a chance to look at this any time soon, @mariocj89 / @tirkarthi do you folks have any time?

@infohash
Copy link
Contributor Author

infohash commented Feb 11, 2024

Hi! Thanks for looking into this. wraps is an argument defined in unittest.mock.Mock. Here's how official documentation defines it:

wraps: Item for the mock object to wrap. If wraps is not None then calling the Mock will pass the call through to the wrapped object (returning the real result). Attribute access on the mock will return a Mock object that wraps the corresponding attribute of the wrapped object (so attempting to access an attribute that doesn’t exist will raise an AttributeError).

While this argument works as intended to be, it is not respected by create_autospec . This patch ensures that create_autospec must respect this argument when auto-speccing objects.

See Details for full explanation.

@bedevere-app
Copy link

bedevere-app bot commented Feb 11, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Feb 19, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@infohash infohash requested a review from tirkarthi February 19, 2024 14:52
@tirkarthi
Copy link
Member

Thanks for catching the behavior change with my patch. PR looks good to me. Please add a NEWS entry for CI. At some point it I guess it would be good to have precedence of return_value, wraps and side_effect documented when all or combinations of them are set.

I would recommend waiting for comments from @cjw296 and @mariocj89 since it's been sometime when I last worked on mock module and your explanation was of good help. Thanks.

@cjw296 cjw296 merged commit 735fc2c into python:main Mar 8, 2024
@cjw296 cjw296 added 3.11 only security fixes 3.12 only security fixes 3.13 bugs and security fixes needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Mar 8, 2024
@miss-islington-app
Copy link

Thanks @infohash for the PR, and @cjw296 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@miss-islington-app
Copy link

Thanks @infohash for the PR, and @cjw296 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@miss-islington-app
Copy link

Sorry, @infohash and @cjw296, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 735fc2cbbcf875c359021b5b2af7f4c29f4cf66d 3.11

@miss-islington-app
Copy link

Sorry, @infohash and @cjw296, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 735fc2cbbcf875c359021b5b2af7f4c29f4cf66d 3.12

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Ubuntu NoGIL Refleaks 3.x has failed when building commit 735fc2c.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1226/builds/1413) and take a look at the build logs.
  4. Check if the failure is related to this commit (735fc2c) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1226/builds/1413

Failed tests:

  • test.test_concurrent_futures.test_shutdown
  • test_eintr

Failed subtests:

  • test_lockf - main.FNTLEINTRTest.test_lockf
  • test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolForkProcessPoolShutdownTest.test_interpreter_shutdown
  • test_flock - main.FNTLEINTRTest.test_flock
  • test_all - test.test_eintr.EINTRTests.test_all

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/test_eintr.py", line 17, in test_all
    script_helper.run_test_script(script)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/support/script_helper.py", line 316, in run_test_script
    raise AssertionError(f"{name} failed")
AssertionError: script _test_eintr.py failed


Traceback (most recent call last):
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 532, in test_lockf
    self._lock(fcntl.lockf, "lockf")
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 517, in _lock
    raise Exception("failed to sync child in %.1f sec" % dt)
Exception: failed to sync child in 300.1 sec


Traceback (most recent call last):
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/test_concurrent_futures/test_shutdown.py", line 50, in test_interpreter_shutdown
    self.assertEqual(out.strip(), b"apple")
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: b'' != b'apple'


Traceback (most recent call last):
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 535, in test_flock
    self._lock(fcntl.flock, "flock")
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 517, in _lock
    raise Exception("failed to sync child in %.1f sec" % dt)
Exception: failed to sync child in 300.4 sec


Traceback (most recent call last):
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 532, in test_lockf
    self._lock(fcntl.lockf, "lockf")
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 517, in _lock
    raise Exception("failed to sync child in %.1f sec" % dt)
Exception: failed to sync child in 300.5 sec


Traceback (most recent call last):
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 535, in test_flock
    self._lock(fcntl.flock, "flock")
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/buildarea/3.x.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 517, in _lock
    raise Exception("failed to sync child in %.1f sec" % dt)
Exception: failed to sync child in 300.3 sec

@cjw296
Copy link
Contributor

cjw296 commented Mar 17, 2024

This should be backported, but I don't have time to manually drive the cherry-picker :'(

@infohash
Copy link
Contributor Author

@cjw296 I will do it using cherry picker. I thought backport was an automated process.

@cjw296
Copy link
Contributor

cjw296 commented Mar 17, 2024

It is, but as you can see in the comments above, it failed this time due to conflicts.

@infohash infohash deleted the fix-issue-75988 branch March 21, 2024 11:12
infohash added a commit to infohash/cpython that referenced this pull request Mar 21, 2024
…hon#115223)

* set default return value of functional types as _mock_return_value

* added test of wrapping child attributes

* added backward compatibility with explicit return

* added docs on the order of precedence

* added test to check default return_value

(cherry picked from commit 735fc2c)
@bedevere-app
Copy link

bedevere-app bot commented Mar 21, 2024

GH-117119 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Mar 21, 2024
infohash added a commit to infohash/cpython that referenced this pull request Mar 21, 2024
…hon#115223)

* set default return value of functional types as _mock_return_value

* added test of wrapping child attributes

* added backward compatibility with explicit return

* added docs on the order of precedence

* added test to check default return_value

(cherry picked from commit 735fc2c)
@bedevere-app
Copy link

bedevere-app bot commented Mar 21, 2024

GH-117124 is a backport of this pull request to the 3.11 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.11 only security fixes label Mar 21, 2024
cjw296 pushed a commit that referenced this pull request Mar 22, 2024
…-115223) (#117119)

gh-75988: Fix issues with autospec ignoring wrapped object (#115223)

* set default return value of functional types as _mock_return_value

* added test of wrapping child attributes

* added backward compatibility with explicit return

* added docs on the order of precedence

* added test to check default return_value

(cherry picked from commit 735fc2c)
cjw296 pushed a commit that referenced this pull request Mar 22, 2024
…-115223) (#117124)

gh-75988: Fix issues with autospec ignoring wrapped object (#115223)

* set default return value of functional types as _mock_return_value

* added test of wrapping child attributes

* added backward compatibility with explicit return

* added docs on the order of precedence

* added test to check default return_value

(cherry picked from commit 735fc2c)
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024
…hon#115223)

* set default return value of functional types as _mock_return_value

* added test of wrapping child attributes

* added backward compatibility with explicit return

* added docs on the order of precedence

* added test to check default return_value
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
…hon#115223)

* set default return value of functional types as _mock_return_value

* added test of wrapping child attributes

* added backward compatibility with explicit return

* added docs on the order of precedence

* added test to check default return_value
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes 3.12 only security fixes 3.13 bugs and security fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants