Improve the docs regarding the migration from imp to importlib

[alexprengere]

Now that the imp removal has landed on main, users migrating to Python3.12 will likely need help to move to importlib.
The current imp docs have tips on how to do just that. Great!

One caveat: the imp.load_source has been removed from the docs a long time ago, now it is only visible is the Python2 version of the docs. So users of imp.load_source cannot rely on the docs to help them migrate, and have to Google this. The first results on stackoverflow are a bit wrong:

• many point to SourceFileLoader(...).load_module(), but this is also deprecated and slated for removal in 3.12 (according to the warning)
• some solutions point to importlib.util.spec_from_file_location, but this does not work with files that do not end with ".py"

The solution that I think is the best translation:

def imp_load_source(module_name, module_path):
    loader = SourceFileLoader(module_name, module_path)
    module = types.ModuleType(loader.name)
    loader.exec_module(module)
    return module

I think it would be beneficial to have that kind of information in the docs. Unfortunately, imp.load_source is not officially documented, but there are several GitHub issues and SO threads discussing how to migrate code to importlib. IMO, we should do one of:

• add it back to the docs, with explanations on how to migrate it to importlib
• just document the migration in the release notes, in the section "Porting to Python 3.12"

What do you think?

Linked PRs

• gh-104212: Add importlib.util.load_source_path() function #105755
• gh-104212: Explain how to port imp code to importlib #105905
• gh-104212: Explain how to port imp.load_source() #105951
• [3.12] gh-104212: Explain how to port imp code to importlib (GH-105905) #105952
• gh-104212: Explain how to port imp.load_source() #105978
• [3.12] gh-104212: Explain how to port imp.load_source() (GH-105978) #106083

[warsaw]

Maybe we need a porting guide in the importlib docs for porting older imp and other APIs to importlib? I don't like that imp module has been deleted but the docs are still there.

[hugovk]

Good idea!

In fact we were just discussing this earlier today and thought about putting a migration guide in both the imp and importlib docs and in the 3.12 porting guide.

cc @vstinner

[terryjreedy]

In current 3.12, 'imp' is no longer present in the module index, which suggests that it is gone.

[vstinner]

I don't know the importlib design, so maybe it doesn't make sense, but... Would it make sense to add a imp_load_source() function to importlib? Maybe under a different name.

[vstinner]

My own use case of load_module(): old code written first for Python 2 using imp then ported to SourceLoader. It's a short script which looks for "test_xxx.py" files and then run them. Stupid but simple pytest-like, good enough for my needs: https://github.com/vstinner/hachoir/blob/0a030e3c8045441a0c30a04475a94f3aadd818ed/runtests.py#L72

I was already annoyed to have to port to them a first date to get rid of imp. So the code should be updated again?

[CAM-Gerlach]

Would it make sense to add a imp_load_source() function to importlib?

Just to note, see, e.g., #58756 for some previous discussion of this.

[vstinner]

The recipe didn't work for me:

• (1) The module has no __file__ variable
• (2) The module was no registered in sys.modules

imp.load_source() defines __file__ and adds the module to sys.modules.

I used this recipe instead:

def load_module(module_name, filename):
    loader = importlib.machinery.SourceFileLoader(module_name, filename)
    module = types.ModuleType(loader.name)
    module.__file__ = filename
    sys.modules[module.__name__] = module
    loader.exec_module(module)
    return module

[vstinner]

Minor step forward: I documented how to replace removed imp.new_module(): 457a459

[brettcannon]

• some solutions point to importlib.util.spec_from_file_location, but this does not work with files that do not end with ".py"

That's actually only true if you don't pass in the loader you need to use as the code has to guess as what you're after. You can use either importlib.util.spec_from_loader() or importlib.util.spec_from_file_location(), they just need different specifics to do the right thing.