Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-30814: Fixed a race condition when import a submodule from a package. #2580

Merged

Conversation

serhiy-storchaka
Copy link
Member

No description provided.

@mention-bot
Copy link

@serhiy-storchaka, thanks for your PR! By analyzing the history of the files in this pull request, we identified @brettcannon, @pitrou and @ericsnowcurrently to be potential reviewers.

Copy link
Contributor

@ncoghlan ncoghlan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general approach looks good to me, but I think a dedicated helper function will be clearer when handling the mod == Py_None case in C, rather than relying on the second module is None check in _find_and_load.

with _ModuleLockManager(name):
return _find_and_load_unlocked(name, import_)
"""Find and load the module."""
_imp.acquire_lock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This consolidation seems a little confusing to me, but I like the idea of simplifying the C code by moving the ModuleNotFoundError generation into Python.

How about adding a dedicated _raise_for_halted_import(name, import_): function that just handles the module is None case?

goto error;
}
else if (mod != NULL) {
if (mod != NULL && mod != Py_None) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the separate helper function, we'd keep the dedicated mod == Py_None branch, but it would be simplified to just calling the new _raise_for_halted_import helper rather than constructing the exception directly in C.

module = sys.modules[name]
if module is None:
_imp.release_lock()
message = ('import of {} halted; '
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would call _raise_for_halted_import(name, import_).

@serhiy-storchaka
Copy link
Member Author

Calling Python method from the C code is not much simpler than generating an exception.

The module is None case isn't worth optimization. This isn't common case and isn't performance critical case since raising and catching exception is slow. Only the case of already imported module is worth optimization by implementing in C. It was my mistake that I implemented the generating an exception on C in 133138a, but my patch didn't touched Python code at all.

Note that the new _find_and_load() is just a second half of old _gcd_import(). All these checks already were implemented in Python, I just moved the code from one Python function to the other.

If you prefer I can withdraw changes in import.c and create separate PR for simplifying C code (move locking and generating an exception to Python code) in master only.

I also can implement the second check in C and keep Python code unchanged. But I think that it is better to implement performancy non-critical parts in Python.

@ncoghlan
Copy link
Contributor

ncoghlan commented Jul 5, 2017

@serhiy-storchaka I think it's specifically the paired negations in mod != NULL && mod != Py_None that I find awkward in the C code, so switching the sense of that check to be mod == NULL || mod == Py_None (and then swapping the if branches accordingly) would resolve the readability problem without having to change the Python code.

@serhiy-storchaka
Copy link
Member Author

Do you want to swap also other branches for fromlist != NULL && fromlist != Py_None and !has_from?

@ncoghlan
Copy link
Contributor

ncoghlan commented Jul 5, 2017

@serhiy-storchaka I switched my review, as coming back to it after a couple of hours, I now suspect my initial confusion was specifically due to reviewing the diff, rather than reading the updated code standalone.

That realisation came mainly from comparing it to the following fromlist check that you mentioned (which I didn't find hard to read at all, but which also didn't have a diff confusing the matter).

@serhiy-storchaka serhiy-storchaka merged commit b4baace into python:master Jul 6, 2017
@serhiy-storchaka serhiy-storchaka deleted the import-double-check branch July 6, 2017 05:09
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Jul 6, 2017
@bedevere-bot
Copy link

GH-2598 is a backport of this pull request to the 3.6 branch.

@serhiy-storchaka
Copy link
Member Author

Thank you for your review @ncoghlan.

serhiy-storchaka added a commit that referenced this pull request Jul 6, 2017
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Jul 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants