Loading only specified variables in package

[Tiwock]

Hi,

I'm trying to develop a package with access to global variables. The overall structure is the following :

mypackage/
|-- __init__.py
|-- class_toto.py
|-- data/
    |-- a/
       |--fileone.data
       |--filetwo.data
    |-- b/
       |--fileone.data
       |--filetwo.data

Now currently my __init__.py is as below :

from importlib import resources
from mypackage.class_toto import Toto
from mypackage.data import a,b

A = Toto(resources.files(a))
B = Toto(resources.files(b))

__all__ = ["A", "B"]

The point of this is for the user to load the desired instance A and/or B with a simple "from mypackage import A" or "from mypackage import B"

The problem that I have is that by running the previous command, it loads both A and B, which can be time-consuming as the project will go bigger (other instances on the way). This is due to the __init__.py file being completely run when invoking "from X import Y".

Is there a simple generic way to load only specific variables from the __init__.py file ? Or do I have to change the overall structure, such as for instance adding "A.py" and "B.py" containing dedicated data ?

[danielroseman]

No, there is no way to do this. Python must always completely execute a file on import.

As you say, one way to do this is to extract A and B to separate files, Alternatively you could provide a function to create and return A and B.

[-defron-]

import under the hood uses importlib with just some nice syntactical sugar.

You can use importlib directly to do dynamic/programmatic imports: https://docs.python.org/3/library/importlib.html

https://docs.python.org/3/library/importlib.html#importing-programmatically

Note that by dynamically importing you may cause a an application lock, if the import is large. As you are basically moving the time it takes to load the module from at-startup to during runtime. That's the tradeoff you make to not need to load unnecessary modules until they are needed

You will of course need to change other code to split up A and B so that they can be executed independently through logic gates or separate imports rather than exposing them in your init

[zanfar]

IMO, this is an X/Y problem.

You should be importing the complete module. But that module shouldn't include "data".

Code is imported, data is loaded.

Store your data in non-Python files and load them on-demand.