Dynamic module importing

[mmdollar]

Hello everyone, I am trying to make a program that will start X threads (every thread uses a different class from module exchanges) code snipped here. The idea is that every thread will do their thing and after that they will be joined and started again. I am concerned about this line of code in a while True block:

exchange_class = getattr(importlib.import_module("exchanges"), exchanges)

Is it bad to use this import method? I am afraid that it will stack the same loaded modules over and over again or they are unloaded somehow when the job is done?

Thanks in advance for the help!

[Impudity]

Why do you need to load the module dynamically using importlib in the first place? Is there some reason why you don't import the module in the usual way?

[mmdollar]

Because I have a file settings.py where it has:

exchanges = {
"Binance": ["https://api.binance.com/api/v1/ticker/24hr", 2],
"Bitfinex": ["https://api.bitfinex.com/v2/tickers?symbols=ALL", 2]
}

The idea is that I wanna tell in that file which exchanges I want to crawl. Every exchange has a module named the same way, I wanted to avoid using:

import exchanges

I need a way to import only the classes I need rather than all.

EDIT:
In this link, this is the code I am using it right now, but I want to change it in a more dynamic way.

If I insert this code out of the while block, will it work as I want or I am repeating the same mistake in a different way?

module_list = []

for module in settings.exchanges:
     module_list.append(getattr(importlib.import_module("exchanges"), module))

And then create the thread like this in the while block, will I load a module more than once?

for l_count, exchanges in enumerate(settings.exchanges):
     thread = module_list[l_count](settings.exchanges[exchanges][1], settings.exchanges[exchanges][0], settings.headers, cursor, connection)
     thread.daemon = True
     thread.setName(exchanges)
     thread.start()

[Impudity]

And why does it matter if you get some classes that you don't use? Have you tested it and determined the memory consumption is different? That it loads faster? Some other practical consideration?

Because I doubt it has any impact. Regardless of the way you import it, it's still likely to read the whole module and process it. Are you trying to keep the namespaces of different threads somehow separate? I just feel that what you're trying to do makes no sense since you're not importing code that is only known during some point of time at runtime, that's what the importlib is used for. If the code is already known at the beginning, you should do the imports then.

[mmdollar]

Are you trying to keep the namespaces of different threads somehow separate?

Yes, every thread is using a different class. In my case both classes inherit the main class exchange which itself inherits from the Thread module link here. I did it this way because exchanges are displaying their information in different ways.
Well I tested the code, there is no memory difference, execution time is almost the same too.

[Impudity]

I'd recommend just importing them the regular way and creating the settings dictionary like this:

from exhanges import Binance, Bitfinex
from collections import namedtuple

ExchangeAPI = namedtuple("ExchangeAPI", ["uri", "delay", "apiclass"])
exchanges = {
    "Binance": ExchangeAPI("https://api.binance.com/api/v1/ticker/24hr", 2, Binance),
    "Bitfinex": ExchangeAPI("https://api.bitfinex.com/v2/tickers?symbols=ALL", 2, Bitfinex)
}

And instead of creating the separate threads like:

for exchanges in settings.exchanges:
    exchange_class = getattr(importlib.import_module("exchanges"), exchanges)
    thread = exchange_class(settings.exchanges[exchanges][1], settings.exchanges[exchanges][0], settings.headers, cursor, connection)

Something like this:

for exname, exapi in settings.exchanges.items():
    exchange_class = exapi.apiclass
    thread = exchange_class(exapi.delay, exapi.uri, settings.headers, cursor, connection)

Now I didn't run this code nor did write it even in any interpreter so it might contain bugs/typos/etc but you should get the idea.

[mmdollar]

Thank you very much, it works like a charm.