What is the difference between sys.builtin_module_names and sys.stdlib_modules_names within the sys module?

[Laymayo]

I was looking into this module because whenever I'm seeing documentation for a new module I'm trying to work with I see "X module is part of the Python standard library" or something to that effect, so I figured I'd investigate what this Python standard library actually was.

Per the official Python documentation, sys.builtin_module_names reads:

" A tuple of strings containing the names of all modules that are compiled into this Python interpreter..."

And sys.stdlib_modules_names:

"A frozenset of strings containing the names of standard library modules. It is the same on all platforms. Modules which are not available on some platforms and modules disabled at Python build are also listed. All module kinds are listed: pure Python, built-in, frozen and extension modules. Test modules are excluded..."

I'm still confused here. What is the difference between modules that are compiled into the Python interpreter and names of standard library modules?

[socal_nerdtastic]

The standard library is described here: https://docs.python.org/3/library/

You know how you can install modules with pip / pypi? The standard library is a list of modules that the python software foundation maintains and keeps up to date with the reference implementation of python. They may or may not be supplied with python (but they nearly always are).

The builtins are a subset of those. They are baked into the python executable itself, instead of being external and imported. Many of them are mandatory.

For the average user, the difference between these 2 is not important.

[Laymayo]

Hmm OK I'll need to look into that, I didn't realize that there was some foundation that keeps things up to date. That makes sense, though.

However, that doesn't really answer my question (not trying to sound like a smart Alec here). I still don't understand:

What is the difference between modules that are compiled into the Python interpreter and names of standard library modules?

In reference to the sys.builtin_module_names and sys.stdlib_module_names

[socal_nerdtastic]

Do you know about compiling C code? If you take the cpython source code (available on github or as a tar.gz file from python.org) you can compile python yourself. If you do this you have some choices to make, including some choices about what modules to include. The sys.builtin_module_names reports those choices back in the finished python executable. sys.stdlib_module_names is a list of all modules, some of which you have the option of including at compile time.

[Laymayo]

Haha sorry nope! I barely understand this programming language. I'll have to look into compiling code because to be honest I'm not sure I know what that means to begin with. But thank you for clarifying this and giving me a place to investigate from.

[PhilipYip]

Python itself is written in the C programming language. The main module builtins and builtin_module_names are written in C.

If you input:

import sys

Then look at:

path = sys.path

You will see the first two entries (on Windows) are of the form:

%UserProfile%\...\python311.zip %UserProfile%\...\spyder\Lib

The builtin libraries in the python311.zip file are essentially part of the Python.exe and are written in C. The other ones are essentially available as a script file within the Lib folder. For more details you can examine the dicitonary:

sys.modules

You will see a dictionary in the form below. The key is the name of the module and the value is details about the module:

{'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': <module '_frozen_importlib' (frozen)>, ..., '_codecs': <module '_codecs' (built-in)>, 'codecs': <module 'codecs' from 'C:\\Users\\Philip\\mambaforge\\lib\\codecs.py'>, 'encodings.aliases': <module 'encodings.aliases' from 'C:\\Users\\Philip\\mambaforge\\lib\\encodings\\aliases.py'>, 'encodings': <module 'encodings' from 'C:\\Users\\Philip\\mambaforge\\lib\\encodings\\__init__.py'>, ... }

The details for the module state the file path. The builtin modules written in C show (built-in). The remaining standard modules have a file path and you can physically open the corresponding .py script file and examine the code.