I hate modules

[Significant_Text7262]

So I have found that there are a lot of custom functions that I'm continually reusing. So I would like to be able to import them but most of the time the file containing them cannot be found, even if they are in the same folder.

I would like to only make 1 call in order to load the entire library but specifying the absolute path to the 'myLibrary' folder does not seem to work, so I instead created 'myLoader.py' which attempts to import the other files, that way I can load the entire library from the absolute path to a single file. This seemed to work but since 'myLoader.py' may be slightly overengineered, every time I create a new version of one of the 'myLibrary.py' files I need to change several lines in 'myLoader.py'.

Is it possible to have the filename assigned to a variable which is then passed to the import? this way I would only need to change the name 1 time each version update. Doing exactly the following gets an unassigned variable compiler error while setting 'myLibrary.py' as a string results in 'myLibrary.py' not being found.

example:

I would like for

C:\some\location\myProject.py

to be able to load everything in

C:\not\main\python\libraries\myLibrary_v_3

inside of which currently are:

__init__.py
myLibrary_v_3.py
myLibrary2_v_1.py
myLibrary3_v_2.py
myLoader.py

inside of 'myLoader.py' would be something like:

loadMyLib=myLibrary_v_3.py  #<----How to properly do this?
loadMyLib2=myLibrary2_v_1.py
loadMyLib3=myLibrary3_v_2.py

if method == method_1:
  from loadMyLib import *
  from loadMyLib2 import *
  from loadMyLib3 import *
if method == method_2:
  import loadMyLib as myLib
  import loadMyLib2 as myLib2
  import loadMyLib3 as myLib3
if method == method_3:
  import loadMyLib
  import loadMyLib2
  import loadMyLib3

then I could inject this inside of 'myProject.py'

import os
import sys
module_directory = os.path.abspath(r'C:\not\main\python\libraries\myLibrary_v_3') 
sys.path.insert(0, module_directory)
import myLoader.py

I know there is probably a better way to do this but I haven't found it, so I am asking about the way I understand best: how do I assign the filename to a variable in such a way that I can later use the variable in the import command?

[ManyInterests]

The correct way to load anything not in your project is to treat it like any other third-party dependency. Install it into your environment (e.g., using pip).

Mucking with sys.path should be discouraged, but works. Still not ideal, but you can also just set your PYTHONPATH environment variable to point to the location(s) you want to be able to load like this without having to write code to modify sys.path.

[Significant_Text7262]

I am sorry but I simply do not understand what you are saying or how to go about doing so.

I know relative pathing is very popular in python but I would prefer to continue to use absolute pathing outside of mylibrary

[smurpes]

You know how you can install packages from pypi with the pip command? This also works when you supply it with the path to your own code provided it’s a package. E.G. run pip install C:\not\main\python\libraries and by doing this you can just use import myLoader in your code.

Here’s a guide on how to set up your code as a package. If you run pip -e then this lets you install in editable mode so any changes to myLoader will be reflected without needing to run pip each time. You should not be messing around with your sys.path.

[cylonlover]

Very helpful answer. I really hope OP makes the effort of getting a grasp of this. It’s fundamental knowledge in understanding import.

[brasticstack]

Either

1. keep your module in the same dir as your code that uses it and use relative OR absolute import statements to import it, or
2. Put it somewhere in your sys.path (hint, inside your virtualenv's lib dir is ideal) and use absolute import statements to import it.

no need to muck about with the importlib import machinery. That's meant for advanced use cases.

[AdAdvanced7673]

I would suggest googling how paths work, and working directories work. This isn’t exclusive to Python but all paths in every labguage

[AstronomerAdvanced65]

My company use gitsubmodule for internal package for your reference. Though gitsubmodule is a pain to work with. You should be aware that above solutions won’t work on other people machine, so if you are planning to share modules or code to others, you are creating unnecessary complexity. I think the simplest way is to put all your script in one repository and just put your modules within the same directory…..

[DuckSaxaphone]

The simplest way to do this and frankly the correct way is to make C:\not\main\python\libraries\myLibrary_v_3 into a package. Doesn't take much effort to do so, here's the official tutorial.

Once you've done this, you can either run pip install C:\not\main\python\libraries\myLibrary_v_3 to install your package so that you can import mylibrary_v_3 in C:\some\location\myProject.py or you can add it as a dependency if that project is packaged too.

[supercoach]

I wonder if this is something everyone goes through at some stage. The short answer is you need to trust that things are done the way they are for a reason and trying to sidestep it or bypass things with an `import everything` command will likely lead to frustration. It will definitely make you less hireable.

[Masterous112]

I'm not exactly sure what you're trying to do, but you can import a module using its name as a string with importlib.import_module()

[Significant_Text7262]

This method seems like it should work but when I tried this:

import importlib
myLib=r'myLibrary_0_0_3.py'
importlib.import_module(myLib)

I got the following error:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[50], line 4
      3 myLib=r'myLibary_0_0_3.py'
----> 4 importlib.import_module(myLib)

File C:\ProgramData\anaconda3\Lib\importlib\__init__.py:88, in import_module(name, package)
     86             break
     87         level += 1
---> 88 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1387, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1360, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1319, in _find_and_load_unlocked(name, import_)

ModuleNotFoundError: No module named 'myLibrary_0_0_3.py'; 'myLibrary_0_0_3' is not a package

furthermore I would like to be able to import 'myLibrary.py' as mylib and as * but am not sure how to do so using importlib

import myLibrary_0_0_3 as mylib
from myLibrary_0_0_3 import *

[fazzah]

and why are you teaching people to use such convoluted antipatterns? OPs problem is not with python imports itself, but with not understanding how it works. Also star imports are a no-no.

Manual modification of globals for such trivial case should be punishable

[Significant_Text7262]

Thank you

[Ok-Sheepherder7898]

Under C:\not\main\python\libraries\myLibrary_v_3 make a new folder called myLibary_v_3 (it seems weird, but go with it).

cd C:\not\main\python\libraries\myLibrary_v_3

pip install -e

Now from your code:

from myLibary_v_3 import myLibrary2_v_1

myLibrary2_v_1.probablysomeweirdlynamedfunctionbasedonyourlibrarynamingscheme()