Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-128911: Add tests on the PyImport C API #128915

Merged
merged 10 commits into from
Jan 17, 2025
Merged

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Jan 16, 2025

  • Add Modules/_testlimitedcapi/import.c
  • Add Lib/test/test_capi/test_import.py
  • Remove _testcapi.check_pyimport_addmodule(): tests already covered by newly added tests.

@vstinner vstinner changed the title gh-128912: Add tests on the PyImport C API gh-128911: Add tests on the PyImport C API Jan 16, 2025
* Add Modules/_testlimitedcapi/import.c
* Add Lib/test/test_capi/test_import.py
* Remove _testcapi.check_pyimport_addmodule(): tests already covered
  by newly added tests.
@vstinner
Copy link
Member Author

cc @serhiy-storchaka @encukou

@vstinner
Copy link
Member Author

Importing asyncio package works if test_capi.test_import is run alone, but fails with the following error when test_capi is run:

ERROR: test_importmodule (test.test_capi.test_import.ImportTests.test_importmodule) (name='asyncio')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_capi/test_import.py", line 42, in check_import_fresh_module
    module = import_module(name)
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/asyncio/__init__.py", line 25, in <module>
    __all__ = (base_events.__all__ +
               ^^^^^^^^^^^
NameError: name 'base_events' is not defined

For now, I modified the tests to not import asyncio.

Avoid the datetime module.
Modules/_testlimitedcapi/import.c Outdated Show resolved Hide resolved
Modules/_testlimitedcapi/import.c Show resolved Hide resolved
Modules/_testlimitedcapi/import.c Show resolved Hide resolved
Modules/_testlimitedcapi/import.c Outdated Show resolved Hide resolved
Modules/_testlimitedcapi/import.c Outdated Show resolved Hide resolved
Modules/_testlimitedcapi/import.c Outdated Show resolved Hide resolved
Modules/_testlimitedcapi/import.c Outdated Show resolved Hide resolved
@vstinner
Copy link
Member Author

I will merge my PR as it is when the CI passed, to be able to add tests in my following #128912 PR.

I didn't add tests for non-UTF8 strings, nor NULL tests for PyImport_ImportModuleLevel(). For PyImport_ImportModuleLevel(), it's tricky because some arguments are ignored for level=0 and I don't know how to write tests for level > 0.

@serhiy-storchaka: Maybe you would be interested to work on follow-up PR to increase the test coverage / test more cases?

Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you allow me to commit directly in your branch? It is better to have a single clean commit in the logs than several commits that rewrite one other.

Modules/_testlimitedcapi/import.c Outdated Show resolved Hide resolved
Modules/_testlimitedcapi/import.c Outdated Show resolved Hide resolved
@vstinner
Copy link
Member Author

@serhiy-storchaka: Ok, you can push changes to my branch.

@serhiy-storchaka
Copy link
Member

I pushed my changes. Please review them.

It seems that I found bugs in the current code:

  1. PyImport_ExecCodeModuleObject() leaves a non-string name in sys.module after raising AttributeError.
  2. There is a difference in __spec__.origin between PyImport_ExecCodeModuleWithPathnames() and PyImport_ExecCodeModuleObject() if the first pathname is NULL and the second pathname is not NULL.

@vstinner
Copy link
Member Author

I pushed my changes. Please review them.

I pushed a cleanup changes. With this cleanup, the change (additional tests) LGTM.

It seems that I found bugs in the current code

Oh, interesting.

@vstinner vstinner enabled auto-merge (squash) January 17, 2025 17:31
@vstinner
Copy link
Member Author

@serhiy-storchaka: Thanks for additional tests and reviews.

I enabled auto-merge.

@vstinner vstinner merged commit d95ba9f into python:main Jan 17, 2025
39 checks passed
@vstinner vstinner deleted the test_capi_import branch January 17, 2025 17:52
@serhiy-storchaka serhiy-storchaka added needs backport to 3.12 bug and security fixes needs backport to 3.13 bugs and security fixes labels Jan 17, 2025
@miss-islington-app
Copy link

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@miss-islington-app
Copy link

Sorry, @vstinner, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker d95ba9fa1110534b7247fa2ff12b90e930c93256 3.13

@miss-islington-app
Copy link

Sorry, @vstinner, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker d95ba9fa1110534b7247fa2ff12b90e930c93256 3.12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs backport to 3.12 bug and security fixes needs backport to 3.13 bugs and security fixes skip news
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants