Skip to content

bpo-42532: Check if NonCallableMock's spec_arg is not None instead of call its __bool__ function #23613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

idanw206
Copy link
Contributor

@idanw206 idanw206 commented Dec 2, 2020

https://bugs.python.org/issue42532

If an object has logic in its __bool__ function, it'll be called here, but it shouldn't be called.

@the-knights-who-say-ni
Copy link

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@idanw206

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

Copy link
Contributor

@mariocj89 mariocj89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely spotted bug!

Co-authored-by: Mario Corchero <mariocj89@gmail.com>
@idanw206
Copy link
Contributor Author

idanw206 commented Dec 4, 2020

Nicely spotted bug!

Thanks! :)

Copy link
Contributor

@cjw296 cjw296 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice spot, but please can you add a test that fails before your fix and passes after it.

@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

And if you don't make the requested changes, you will be poked with soft cushions!

@idanw206 idanw206 force-pushed the bpo-42532-dont-call-bool-on-noncallablemock-specarg branch from a4c2be1 to 6ac0e20 Compare December 5, 2020 17:51
@idanw206
Copy link
Contributor Author

idanw206 commented Dec 5, 2020

I have made the requested changes; please review again.

@bedevere-bot
Copy link

Thanks for making the requested changes!

@cjw296: please review the changes made to this pull request.

@bedevere-bot bedevere-bot requested a review from cjw296 December 5, 2020 17:52
@idanw206 idanw206 force-pushed the bpo-42532-dont-call-bool-on-noncallablemock-specarg branch from 6ac0e20 to 71c4f0c Compare December 6, 2020 08:24
@idanw206 idanw206 force-pushed the bpo-42532-dont-call-bool-on-noncallablemock-specarg branch from 3ad46ec to 190c9ed Compare December 6, 2020 09:19
@miss-islington
Copy link
Contributor

Thanks @idanw206 for the PR, and @cjw296 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Thanks @idanw206 for the PR, and @cjw296 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry, @idanw206 and @cjw296, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker c598a04dd29b89ad072245ddaf738badcfb41ac7 3.8

@miss-islington
Copy link
Contributor

Sorry @idanw206 and @cjw296, I had trouble checking out the 3.9 backport branch.
Please backport using cherry_picker on command line.
cherry_picker c598a04dd29b89ad072245ddaf738badcfb41ac7 3.9

@cjw296
Copy link
Contributor

cjw296 commented Dec 6, 2020

@tirkarthi - I've taken off the backport labels since neither of the ones I added appear to have worked. Where should this get backported and how do we do that nowadays?

@tirkarthi
Copy link
Member

There is a tool called cherry_picker that needs to be installed and can be run locally to cherry pick the commits. It seems bot cannot do it automatically due to conflicts. I will raise manual backports.

Dev guide : https://devguide.python.org/committing/?highlight=Backport#backporting-changes-to-an-older-version

@cjw296
Copy link
Contributor

cjw296 commented Dec 6, 2020

I guess I'm concerned that there are conflicts, it doesn't feel like there should be?

@tirkarthi
Copy link
Member

tirkarthi commented Dec 6, 2020

Yes, it might affect mock backport. The change is simple enough so not sure if conflict is in mock.py or the tests. I will update here once I try the backport to 3.9 and 3.8.

@tirkarthi
Copy link
Member

tirkarthi commented Dec 7, 2020

@cjw296 I get below patch as conflict while trying to cherry pick. I guess 3.8 doesn't need this since it seems to iterate through all args and check for spec which has been simplified in 3.9 and above with #16099. The simplification has a backport to 3.8 in #16137 but was not applied. Hence backporting this PR would need #16137 also to be applied.

diff --cc Lib/unittest/mock.py
index 3629cf6109,4db1bacf4b..0000000000
--- a/Lib/unittest/mock.py
+++ b/Lib/unittest/mock.py
@@@ -403,18 -402,12 +403,25 @@@ class NonCallableMock(Base)
          # so we can create magic methods on the
          # class without stomping on other mocks
          bases = (cls,)
 -        if not issubclass(cls, AsyncMockMixin):
 +        if not issubclass(cls, AsyncMock):
              # Check if spec is an async object or function
++<<<<<<< HEAD
 +            sig = inspect.signature(NonCallableMock.__init__)
 +            bound_args = sig.bind_partial(cls, *args, **kw).arguments
 +            spec_arg = [
 +                arg for arg in bound_args.keys()
 +                if arg.startswith('spec')
 +            ]
 +            if spec_arg:
 +                # what if spec_set is different than spec?
 +                if _is_async_obj(bound_args[spec_arg[0]]):
 +                    bases = (AsyncMockMixin, cls,)
++=======
+             bound_args = _MOCK_SIG.bind_partial(cls, *args, **kw).arguments
+             spec_arg = bound_args.get('spec_set', bound_args.get('spec'))
+             if spec_arg is not None and _is_async_obj(spec_arg):
+                 bases = (AsyncMockMixin, cls)
++>>>>>>> c598a04dd2... [bpo-42532](https://bugs.python.org/issue42532): Check if NonCallableMock's spec_arg is not None instead of call its __bool__ function (GH23613)
          new = type(cls.__name__, bases, {'__doc__': cls.__doc__})
          instance = _safe_super(NonCallableMock, cls).__new__(new)
          return instance

@cjw296
Copy link
Contributor

cjw296 commented Dec 7, 2020

It's unclear to me why #16099 doesn't have any tests as part of the commit. That would help us figure out whether both or neither of these PRs should end up in 3.8...

@tirkarthi
Copy link
Member

My guess is that it's a refactoring PR where all bound_args are iterated and checking for starting with "spec" might give others starting with spec like "specfoo" which could have been neither supported/tested. The PR probably tries to be explicit about checking for only spec_set and spec.

@cjw296
Copy link
Contributor

cjw296 commented Dec 10, 2020

@idanw206 - this has been released in the backport now: https://pypi.org/project/mock/4.0.3/

tirkarthi added a commit that referenced this pull request Dec 14, 2020
…ead of call its __bool__ function (GH-23613) (GH-23676)

Check if NonCallableMock's spec_arg is not None instead of call its __bool__ function
(cherry picked from commit c598a04)

Co-authored-by: idanw206 <31290383+idanw206@users.noreply.github.com>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora LTO + PGO 3.9 has failed when building commit 14f2a12.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/423/builds/358) and take a look at the build logs.
  4. Check if the failure is related to this commit (14f2a12) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/423/builds/358

Summary of the results of the build (if available):

Click to see traceback logs
remote: Enumerating objects: 1        
remote: Enumerating objects: 12, done.        
remote: Counting objects:   8% (1/12)        
remote: Counting objects:  16% (2/12)        
remote: Counting objects:  25% (3/12)        
remote: Counting objects:  33% (4/12)        
remote: Counting objects:  41% (5/12)        
remote: Counting objects:  50% (6/12)        
remote: Counting objects:  58% (7/12)        
remote: Counting objects:  66% (8/12)        
remote: Counting objects:  75% (9/12)        
remote: Counting objects:  83% (10/12)        
remote: Counting objects:  91% (11/12)        
remote: Counting objects: 100% (12/12)        
remote: Counting objects: 100% (12/12), done.        
remote: Compressing objects:  16% (1/6)        
remote: Compressing objects:  33% (2/6)        
remote: Compressing objects:  50% (3/6)        
remote: Compressing objects:  66% (4/6)        
remote: Compressing objects:  83% (5/6)        
remote: Compressing objects: 100% (6/6)        
remote: Compressing objects: 100% (6/6), done.        
remote: Total 12 (delta 6), reused 6 (delta 6), pack-reused 0        
From https://github.com/python/cpython
 * branch                  3.9        -> FETCH_HEAD
Reset branch '3.9'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [Makefile:1855: clean-retain-profile] Error 1 (ignored)
In function ‘assemble_lnotab’,
    inlined from ‘assemble_emit’ at Python/compile.c:5717:25,
    inlined from ‘assemble’ at Python/compile.c:6056:18:
Python/compile.c:5671:19: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
 5671 |         *lnotab++ = k;
      |         ~~~~~~~~~~^~~
In function ‘assemble_lnotab’,
    inlined from ‘assemble_emit’ at Python/compile.c:5717:25,
    inlined from ‘assemble’ at Python/compile.c:6056:18:
Python/compile.c:5671:19: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
 5671 |         *lnotab++ = k;
      |                   ^
In function ‘assemble_lnotab’,
    inlined from ‘assemble_emit’ at Python/compile.c:5717:25,
    inlined from ‘assemble’ at Python/compile.c:6056:18:
Python/compile.c:5671:19: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
 5671 |         *lnotab++ = k;
      |                   ^
In function ‘assemble_lnotab’,
    inlined from ‘assemble_emit’ at Python/compile.c:5717:0,
    inlined from ‘assemble’ at Python/compile.c:6056:0:
Python/compile.c:5671: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
 5671 |         *lnotab++ = k;
      | 
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Modules/_pickle.c: In function ‘load_int’:
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Modules/_pickle.c:5181:17: warning: ‘s’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 5181 |         value = PyLong_FromString(s, NULL, 0);
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Modules/socketmodule.c: In function ‘getsockaddrarg’:
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Modules/socketmodule.c:2395:9: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
 2395 |         strncpy((char *)sa->salg_name, name, sizeof(sa->salg_name));
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Parser/listnode.c: In function ‘list1node’:
Parser/listnode.c:71:1: warning: ‘/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Parser/listnode.gcda’ profile count data file not found [-Wmissing-profile]
   71 | }
      | ^
Python/frozenmain.c: In function ‘Py_FrozenMain’:
Python/frozenmain.c:129:1: warning: ‘/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/frozenmain.gcda’ profile count data file not found [-Wmissing-profile]
  129 | }
      | ^
Python/pyfpe.c: In function ‘PyFPE_dummy’:
Python/pyfpe.c:15:1: warning: ‘/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/pyfpe.gcda’ profile count data file not found [-Wmissing-profile]
   15 | }
      | ^
/usr/bin/ld: python.lto.o: in function `run_mod':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/pythonrun.c:1230: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: python.lto.o: in function `symtable_add_def_helper':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1018: undefined reference to `_Py_Mangle'
/usr/bin/ld: python.lto.o: in function `builtin_compile':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/bltinmodule.c:808: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: python.lto.o: in function `Py_CompileStringObject':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/pythonrun.c:1313: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: python.lto.o: in function `symtable_lookup':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1004: undefined reference to `_Py_Mangle'
/usr/bin/ld: python.lto.o: in function `symtable_record_directive':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1158: undefined reference to `_Py_Mangle'
/usr/bin/ld: python.lto.o: in function `_Py_compile_string':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/./Modules/_peg_parser.c:74: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: python.lto.o: in function `type_new':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Objects/typeobject.c:2532: undefined reference to `_Py_Mangle'
/usr/bin/ld: python.lto.o: in function `symtable_visit_expr':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1018: undefined reference to `_Py_Mangle'
/usr/bin/ld: python.lto.o:/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1018: undefined reference to `_Py_Mangle'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:602: python] Error 1
make[1]: *** Waiting for unfinished jobs....
/usr/bin/ld: _testembed.lto.o: in function `run_mod':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/pythonrun.c:1230: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: _testembed.lto.o: in function `builtin_compile':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/bltinmodule.c:808: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: _testembed.lto.o: in function `Py_CompileStringObject':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/pythonrun.c:1313: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: _testembed.lto.o: in function `symtable_lookup':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1004: undefined reference to `_Py_Mangle'
/usr/bin/ld: _testembed.lto.o: in function `symtable_record_directive':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1158: undefined reference to `_Py_Mangle'
/usr/bin/ld: _testembed.lto.o: in function `_Py_compile_string':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/./Modules/_peg_parser.c:74: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: _testembed.lto.o: in function `type_new':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Objects/typeobject.c:2532: undefined reference to `_Py_Mangle'
/usr/bin/ld: _testembed.lto.o: in function `symtable_add_def_helper':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1018: undefined reference to `_Py_Mangle'
/usr/bin/ld: _testembed.lto.o: in function `symtable_visit_expr':
/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1018: undefined reference to `_Py_Mangle'
/usr/bin/ld: _testembed.lto.o:/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1018: undefined reference to `_Py_Mangle'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:737: Programs/_testembed] Error 1
make: *** [Makefile:545: profile-opt] Error 2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants