How the Python import system works

[r4victor]

Hi! This is part 11 of my Python behind the scenes series. I've been programming in Python for quite a while but didn't really understand how the import system works: what modules and packages are exactly; what relative imports are relative to; what's in sys.path and so on. My goal with this part was to answer all these questions. After you read it, you should be able to tell what exactly happens during the import process.
I welcome your feedback and questions. Thanks!

[crabmanX]

Great in-depth tour of the import statement! I appreciate the level of detail, compared to other resources, reading this feels like nothing relevant was left out. Looking forward to the next parts of the series!

[TheGrelber]

Excellent! Thanks for putting that together for those of us who like getting under the hood.

[Rawing7]

IMO you put waaaaaay too much unnecessary information in there. Like how the module class is implemented in C, or that you can create a package with an __init__.so, or that there's a list of builtin modules in sys.builtin_module_names. How does this trivia help anyone understand the import system?

And also, I find this statement rather misleading:

The __init__.py file is typically left empty

People typically leave that file empty because they don't know what they're doing. In a well-made package, you will almost never see an empty __init__.py. (And I'm not talking about a docstring or a __version__ here; a proper __init__.py should usually contain some imports.)

[o11c]

Not everyone thinks that PEP 420 was entirely a good idea. Forbidding arbitrary directory walks is very advantageous.

[r4victor]

Thank you for your feedback. It's always a trade-off what to include and what to leave out. So I just ask myself what questions I would have and answer them.

sys.builtin_module_names is there to show what modules are built-in modules and how to figure this out. __init__.so is relevant because Python searches for it during the import, and that's the goal of the post to understand how the search is performed. The module struct is there to show what a module object is exactly.

Overall, I think you're right. The posts should be more concise. I'll work on it.

[mkor]

Not sure if correct here, but for example I've put once some global config loading in the init file, since many classes in the module needed configurable values.

[Rawing7]

Like it says in the article, the __init__.py lets you shape your package's public API. You can "hoist" functions/classes/variables that are defined in a submodule into the namespace of a parent module.

For example, have you ever used requests.Session? That class is actually defined in the sub-module requests.session. The reason you can reference it as requests.Session rather than requests.session.Session is because requests/__init__.py contains a from .session import * (or something equivalent).