importorskip decorator on one test skips entire module

[d-chambers]

Bud Description

When using pytest.importorskip as a decorator on a specific test in a test file, the all of the the tests in the file, rather than just the decorated test, are skipped.

For example:

# contents of test_it,py
import pytest

@pytest.importorskip('a_module_that_isnt_installed')
def tests_one():
    assert 1

def test_two():
   assert 0

when running:

pytest test_it.py -rsx

I get the following output:

============================= test session starts ==============================
platform linux -- Python 3.10.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/derrick/python_scratch_space/importorskip
collected 0 items / 1 skipped                                                  

=========================== short test summary info ============================
SKIPPED [1] test_it.py:3: could not import 'a_module_that_isnt_installed': No module named 'a_module_that_isnt_installed'
============================== 1 skipped in 0.00s ==============================

However, I would expect test_two to still be collected and run.

Version Info

Package    Version
---------- -------
attrs      21.4.0
iniconfig  1.1.1
packaging  21.3
pip        21.3.1
pluggy     1.0.0
py         1.11.0
pyparsing  3.0.7
pytest     6.2.5
setuptools 60.5.0
toml       0.10.2
wheel      0.37.1

pytest 6.2.5,
python 10.2
ubuntu 20.04

[RonnyPfannschmidt]

Importorskip is a replacement for import statements, not a marker

[The-Compiler]

From an user's perspective, this is somewhat similar to using pytest.skip at module level - however, there it was possible for us to disallow this in favor of pytestmark = pytest.mark.skip.

Here, I don't think it's possible to do anything about it. importorskip can't know if it was called as part of a decorator or to indeed correctly replace an import statement.

[RonnyPfannschmidt]

maybe it is time to rename it to import_or_skip_whole_module and deprecating the old name indefinitely

[Zac-HD]

importorskip can't know if it was called as part of a decorator

I mean... we could inspect the calling frame, grab the source code, and if "@pytest.importorskip(" in source: raise error. Obviously there are circumstances where this doesn't work, but it's no more dangerous than assertion rewriting and would provide a sensible error + message for this particular misuse.

[RonnyPfannschmidt]

personally i'd prefer just deprecating a easy to misuse api instead of hacking it up with magic to somehow hopefully be able to error

[nicoddemus]

I agree with not adding special magic just to have a better error message, as I think this is a slipper slope: after adding it, users then report a case where our magic didn't work, then we tweak it further (possibly adding a different variant say if "importorskip"), then we get error reports because our magic misbehaved for their use case (issuing a warning when it shouldn't), and so on.

Closing for now. If we really want to do a followup action, I suggest we create a separate issue. 👍

[d-chambers]

Hi thanks guys. I see my issue now for sure, I was using it in what felt like an intuitive way rather than reading the docs 😜. I think a warning on the doc page would be sufficient to help others in my shoes figure out what is wrong more easily.

Also, how would you feel about a built in importorskip mark for just this type of use?

[nicoddemus]

Also, how would you feel about a built in importorskip mark for just this type of use?

Not sure, perhaps it would bring more confusion than help out? We see users confusing pytest.mark.skip and pytest.skip already... 🤔