AttributeError: Test modules cannot use parent module's attributes under `--import-mode=importlib` #10337

akhilramkee · 2022-10-04T17:19:25Z

a detailed description of the bug or problem you are having
output of pip list from the virtual environment you are using
pytest and operating system versions
minimal example if possible

Description of the bug:
Pytest when executed under --import-mode=importlib doesn't try to retain the child module as parent module's attribute if parent module is already in sys.path. This started happening in pytest 7.1.3 after enabling doctest to respect import-mode in #10088.

Minimal Example

File structure

pytest_importlib_issue/
├── __init__.py
├── temp.py
└── tests/
    └── test_temp.py

temp.py

def x():
    """
    >>> assert 1==1
    """
    return 2

test_temp.py

import pytest_importlib_issue
from pytest_importlib_issue import temp

def test_x():
    assert temp.x() == 2
    assert pytest_importlib_issue.temp.x() == 2

Output - MacOS 12.6

pytest7.1.3 without importlib

(pytest7.1.3) akhileshr@Akhileshs-MacBook-Air /tmp % pytest pytest_importlib_issue --doctest-modules                                       
============================================================================================================ test session starts =============================================================================================================
platform darwin -- Python 3.9.10, pytest-7.1.3, pluggy-0.13.1
rootdir: /private/tmp
plugins: anyio-3.5.0, xdist-2.2.1, easyread-0.1.0, forked-1.3.0
collected 2 items                                                                                                                                                                                                                            

pytest_importlib_issue/temp.py .                                                                                                                                                                                                       [ 50%]
pytest_importlib_issue/tests/test_temp.py .                                                                                                                                                                                            [100%]

============================================================================================================= 2 passed in 0.03s ==============================================================================================================

pytest-7.1.3 with importlib

(pytest7.1.3) akhileshr@Akhileshs-MacBook-Air /tmp % pytest pytest_importlib_issue --import-mode=importlib --doctest-modules                                             
============================================================================================================ test session starts =============================================================================================================
platform darwin -- Python 3.9.10, pytest-7.1.3, pluggy-0.13.1
rootdir: /private/tmp
plugins: anyio-3.5.0, xdist-2.2.1, easyread-0.1.0, forked-1.3.0
collected 2 items                                                                                                                                                                                                                            

pytest_importlib_issue/temp.py .                                                                                                                                                                                                       [ 50%]
pytest_importlib_issue/tests/test_temp.py F                                                                                                                                                                                            [100%]

================================================================================================================== FAILURES ==================================================================================================================
___________________________________________________________________________________________________________________ test_x ___________________________________________________________________________________________________________________

    def test_x():
        assert temp.x() == 2
>       assert pytest_importlib_issue.temp.x() == 2
E       AttributeError: module 'pytest_importlib_issue' has no attribute 'temp'

pytest_importlib_issue/tests/test_temp.py:6: AttributeError
========================================================================================================== short test summary info ===========================================================================================================
FAILED pytest_importlib_issue/tests/test_temp.py::test_x - AttributeError: module 'pytest_importlib_issue' has no attribute 'temp'
======================================================================================================== 1 failed, 1 passed in 0.08s =========================================================================================================

It could be seen that the failure happens when we try to access the child module through parent module.

pytest-7.0.1 with importlib

(pytest7.0.1) akhileshr@Akhileshs-MacBook-Air /tmp % pytest pytest_importlib_issue --import-mode=importlib --doctest-modules
============================================================================================================ test session starts =============================================================================================================
platform darwin -- Python 3.9.10, pytest-7.0.1, pluggy-0.13.1
rootdir: /private/tmp
plugins: anyio-3.5.0, xdist-2.2.1, easyread-0.1.0, forked-1.3.0
collected 2 items                                                                                                                                                                                                                            

pytest_importlib_issue/temp.py .                                                                                                                                                                                                       [ 50%]
pytest_importlib_issue/tests/test_temp.py .                                                                                                                                                                                            [100%]

============================================================================================================= 2 passed in 0.03s ==============================================================================================================

This passes, because doctest-modules were using prepend instead of importlib before the PR. ( added this to state this is a recent issue ).

Fix: Child module has to be persisted in Parent module as an attribute similar to how importlib.import_module does.

The text was updated successfully, but these errors were encountered:

akhilramkee · 2022-10-04T17:23:13Z

Have the required fix lined up in #10338. Please let me know if it looks good or I am missing something.

Thanks!

akhilramkee · 2022-11-01T04:41:54Z

any thoughts on this ? @nicoddemus / @RonnyPfannschmidt / @bluetech

jaraco · 2023-06-29T18:34:24Z

This issue also affects pmxbot.

I created an even smaller repro with one namespace package, one submodule, and one test:

 draft @ tree
.
├── pkg
│   └── mod.py
└── tests
    └── test_something.py

3 directories, 2 files
 draft @ cat pkg/mod.py
 draft @ cat tests/test_something.py
import pkg.mod


def test_mod():
    pkg.mod

 draft @ pip-run pytest -- -m pytest --doctest-modules --import-mode importlib
========================================================================= test session starts ==========================================================================
platform darwin -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /Users/jaraco/draft
plugins: anyio-3.6.2
collected 1 item                                                                                                                                                       

tests/test_something.py F                                                                                                                                        [100%]

=============================================================================== FAILURES ===============================================================================
_______________________________________________________________________________ test_mod _______________________________________________________________________________

    def test_mod():
>       pkg.mod
E       AttributeError: module 'pkg' has no attribute 'mod'

tests/test_something.py:5: AttributeError
======================================================================= short test summary info ========================================================================
FAILED tests/test_something.py::test_mod - AttributeError: module 'pkg' has no attribute 'mod'
========================================================================== 1 failed in 0.02s ===========================================================================

Remove --doctest-modules or --import-mode importlib or move tests/test_something.py to . and the test passes.

Also confirmed that the bugfix proposed in #10338 fixes the issue:

 draft @ pip-run 'pytest@git+https://github.com/akhilramkee/pytest@importlib-parent' -- -m pytest --import-mode importlib --doctest-modules
========================================================================= test session starts ==========================================================================
platform darwin -- Python 3.11.4, pytest-7.2.0.dev265+g9118b95f, pluggy-1.2.0
rootdir: /Users/jaraco/draft
plugins: anyio-3.6.2
collected 1 item                                                                                                                                                       

tests/test_something.py .                                                                                                                                        [100%]

========================================================================== 1 passed in 0.01s ===========================================================================

jaraco · 2023-06-29T19:49:10Z

I've also confirmed that with this fix, pmxbot is able to adopt native namespace packages without the test suite failing (pmxbot/pmxbot#110).

Thanks Akhil for the report and fix.

nicoddemus · 2023-07-01T14:48:42Z

any thoughts on this ? @nicoddemus / @RonnyPfannschmidt / @bluetech

Really sorry, I seem to have missed this.

Yeah looks great, it is only missing a changelog entry which I will push up shortly.

Would you like to have your name added to AUTHORS as well @akhilramkee ?

Failing to add child modules as attributes of parent module will prevent them from being accessible through parent module. Fix #10337

akhilramkee · 2023-07-01T18:54:21Z

Yeah looks great, it is only missing a changelog entry which I will push up shortly.

Thanks @nicoddemus for taking care of the missing changelog.

Would you like to have your name added to AUTHORS as well @akhilramkee ?

Sure, I would like that. I can add myself to the AUTHORS in a separate PR if that works.

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [pytest](https://docs.pytest.org/en/latest/) ([source](https://github.com/pytest-dev/pytest), [changelog](https://docs.pytest.org/en/stable/changelog.html)) | patch | `==7.4.0` -> `==7.4.2` | --- ### Release Notes <details> <summary>pytest-dev/pytest (pytest)</summary> ### [`v7.4.2`](https://github.com/pytest-dev/pytest/releases/tag/7.4.2): pytest 7.4.2 (2023-09-07) [Compare Source](pytest-dev/pytest@7.4.1...7.4.2) ### Bug Fixes - [#11237](pytest-dev/pytest#11237): Fix doctest collection of `functools.cached_property` objects. - [#11306](pytest-dev/pytest#11306): Fixed bug using `--importmode=importlib` which would cause package `__init__.py` files to be imported more than once in some cases. - [#11367](pytest-dev/pytest#11367): Fixed bug where `user_properties` where not being saved in the JUnit XML file if a fixture failed during teardown. - [#11394](pytest-dev/pytest#11394): Fixed crash when parsing long command line arguments that might be interpreted as files. ### Improved Documentation - [#11391](pytest-dev/pytest#11391): Improved disclaimer on pytest plugin reference page to better indicate this is an automated, non-curated listing. ### [`v7.4.1`](https://github.com/pytest-dev/pytest/releases/tag/7.4.1): pytest 7.4.1 (2023-09-02) [Compare Source](pytest-dev/pytest@7.4.0...7.4.1) ## Bug Fixes - [#10337](pytest-dev/pytest#10337): Fixed bug where fake intermediate modules generated by `--import-mode=importlib` would not include the child modules as attributes of the parent modules. - [#10702](pytest-dev/pytest#10702): Fixed error assertion handling in `pytest.approx` when `None` is an expected or received value when comparing dictionaries. - [#10811](pytest-dev/pytest#10811): Fixed issue when using `--import-mode=importlib` together with `--doctest-modules` that caused modules to be imported more than once, causing problems with modules that have import side effects. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.apud.pl/jacek/adventofcode/pulls/32 Co-authored-by: Renovate <renovate@apud.pl> Co-committed-by: Renovate <renovate@apud.pl>

akhilramkee mentioned this issue Oct 4, 2022

Add child modules as attributes of parent modules. #10338

Merged

Zac-HD added type: bug problem that needs to be addressed topic: collection related to the collection phase labels Oct 16, 2022

jaraco mentioned this issue Dec 22, 2022

[doctests] Namespace packages (PEP 420) missing, leading to errors with relative imports #3396

Closed

jaraco mentioned this issue Jun 29, 2023

Fix duplicated imports with importlib mode and doctest-modules #11148

Merged

nicoddemus closed this as completed in #10338 Jul 1, 2023

nicoddemus pushed a commit that referenced this issue Jul 1, 2023

Add child modules as attributes of parent modules. (#10338)

2f7415c

Failing to add child modules as attributes of parent module will prevent them from being accessible through parent module. Fix #10337

akhilramkee mentioned this issue Jul 1, 2023

Added Akhilesh Ramakrishnan to AUTHORS #11153

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AttributeError: Test modules cannot use parent module's attributes under `--import-mode=importlib` #10337

AttributeError: Test modules cannot use parent module's attributes under `--import-mode=importlib` #10337

akhilramkee commented Oct 4, 2022

akhilramkee commented Oct 4, 2022

akhilramkee commented Nov 1, 2022

jaraco commented Jun 29, 2023

jaraco commented Jun 29, 2023

nicoddemus commented Jul 1, 2023

akhilramkee commented Jul 1, 2023

AttributeError: Test modules cannot use parent module's attributes under --import-mode=importlib #10337

AttributeError: Test modules cannot use parent module's attributes under --import-mode=importlib #10337

Comments

akhilramkee commented Oct 4, 2022

akhilramkee commented Oct 4, 2022

akhilramkee commented Nov 1, 2022

jaraco commented Jun 29, 2023

jaraco commented Jun 29, 2023

nicoddemus commented Jul 1, 2023

akhilramkee commented Jul 1, 2023

AttributeError: Test modules cannot use parent module's attributes under `--import-mode=importlib` #10337

AttributeError: Test modules cannot use parent module's attributes under `--import-mode=importlib` #10337