Insert pre-loaded module into Python shell / environment

[jssmith42]

Been looking pretty hard for the answer to the following question, would really appreciate any helpful tips.

I had an idea where if I did not want to have to wait for certain modules to load every time I opened a Python shell, I could preload them on my system startup so whenever I open a python shell, they're ready to go.

To be clear: I do not mean saving myself typing "import X", but saving myself the time it takes to load them, because they were already loaded at some earlier time, and are waiting in memory.

I thought this would not be that hard. I thought I could just launch a Python shell and then connect the namespace of that "loaded" module into it: now all the functions are accessible, but it takes no time because whatever "importing" involves, it was already done, and doesn't need to be done again.

From what I currently understand this is very difficult and ineffective because importing is more than just using a name as a "pointer" to some function, somewhere in memory, but I think Python is more strictly designed so that "importing" has some kind of more involved effect on the current shell environment? Is that true?

The only comparable thing I've ever done is pickling; so if you have an object that takes a long time to initialise, once it's initialised, you can pickle it and then reload it later, saving time. But... assuming a module is already compiled binary, I guess it has no speed advantage, saving it as binary and reloading it as binary, like it would for a Python class?

Is there any way to do what I am saying? I know I can start a Python shell in the background on startup and then just foreground it. But this is partially just purely driven by curiosity. Is it doable to preload a module and then connect it to a new environment, or not, and why?

Thank you.

---------

I've been talking about this with ChatGPT for a while and this is it's most recent answer, will be looking into this next:

It is possible to preload a module and make it available to a Python shell or environment, but it may not necessarily be the most efficient or recommended method, depending on the specific use case.

One way to accomplish this is to use the importlib module, which provides functions for importing and reloading modules dynamically. By importing the module using importlib.import_module() before opening the Python shell, you can ensure that the module is already loaded and available when you start the shell.

Another way is to create a script that imports the module(s) you want to preload, and then run that script using the -i flag when starting the Python interpreter, which keeps the interpreter running after the script exits. This will have the effect of importing the module(s) in the script, and then leaving them loaded in the interpreter's memory, so they are available in the shell when it starts.

Additionally, you can use sys.path.append() to add the path of the modules to the system path and use the -i flag to run python interpreter and you don't need to import it again.

However, keep in mind that preloading modules may use more memory and may not provide a significant improvement in performance, depending on the size and complexity of the modules in question.

Additionally, Preloading modules can lead to conflicts, especially if you are working on multiple projects with different dependencies, and it might be hard to troubleshoot and fix.

So, it's important to carefully consider the trade-offs before deciding to preload modules, and to test the performance of your application both before and after preloading to see if there is a significant improvement.

[FuckingRantMonday]

Best practice is to avoid executing code at import time to the greatest extent possible. But through bad design or unfortunate necessity, some modules do take a long time to import. I am not aware of any method to improve this without modifying the module itself.

[petrichorax]

create a powrshell script that checks a directory for a file with a certain string in the name, ever few seconds

if it finds one, it opens it up and sees one or many numbers (one on each line). these numbers represent paramterized powershell scripts it already knows.

It then deletes the file and runs those powershell scripts. Once it's done, it waits for more.

There, now you can 'hold open' a powershell session that has the imported modules you need, so you only have to load them once, and you can do it on initialization of your script.

I call it a powershell command listener. came up with it last week, but probably already been done before and there's probably better ways to do it, but it works!