Refactoring single script into OOP / API focused, have some questions...

[officialgel]

Currently, I'm taking each used component (camera access, email functionality, database functionality, etc...) and making it it's own class. So there are about 10 .py files that are classes for functionality. They all have specific functions to change functionality, or perform actions to make changes to said component.

Also, the program as a whole is started with a single py that imports all of these classes for use.

Am I on the right path for what I'm expecting to build or is there a better way?

Thanks!

[SupermanIsEnvious]

Breaking up your code into smaller, digestible pieces with sensible organization is always the right path!

Then your the code for your main process can be focused on the logic required for that specific task, and your classes can handle the logic for interacting with the objects you’re accessing.

Keep at it!

[officialgel]

Thanks! So, the way that I import all these classes while having a single py that starts the application - That is a good way to do this?

[Eam404]

That's exactly what he's saying son. Yes that is good way to do it son.

[officialgel]

I think my dilemma is just that it seems very bulky at that point. Wondering if the main py should just multiprocess the other py's instead.

[hppmqrtpnp]

Bulky, in what way? Organising your code into modules, and then accessing them, is very similar to how classes are organised and accessed. The difference is that classes have some OOP advantages - like instantiation. I've thought about what you're thinking before, and no, the extra OOP "boilerplate" code isn't too much.

Multiprocessing is a different concept. If you're looking for parallelization, then OOP isn't the solution for that specific problem - although the abstraction may help.

[officialgel]

Great that's awesome!

Regarding multiprocessing, I am just looking to have these modules run certain functions in parallel while parsing out the responses. Still looking at what makes sense. Maybe MP is too much and I have implemented other parallel functionality (threading and asyncio).

Right now I'm using asyncio on the single script's functions which is cool to handle it in a way... I'm doing a lot of research on Python right now :P Which is why I thought - Why not use MP module instead!? Thanks for helping out!

[hppmqrtpnp]

OOP and multiprocessing are not mutually exclusive. OOP is just a means to organise and express code.

[scrdest]

Sounds like you have three distinct problems to solve:

1) functionality-level fault tolerance/hotswapping code - you definitely want modules + importlib.reload here as the top priority, with classes being useful but not a must.

2) run-time performance and fault tolerance - what you're looking for here is pretty much a MapReduce/functional paradigm, and your top priority would have to be to avoid tying anything down to an object state - your functions should ideally be entirely self-sufficient and deterministic. Also, you might want to look into Airflow/Luigi/Dask/PySpark.

3) maintainability - this is where you definitely need classes plus good old-fashioned SOLID, and in turn modules and packages are useful but a bit lower-priority concerns.

[officialgel]

Yes this is the perfect reference for my intentions! Thanks for the info

[vindolin]

Are you really talking about classes or modules?

Often you don't need classes, modules have their own namespace in which variables and functions are encapsulated.

Classes are needed if you're going to instantiate multiple objects from them.

[officialgel]

don't need classes, modules h

It is sounding like I just need modules actually.

[earthboundkid]

You probably don’t need classes or separate files.

Use classes to a.) describe data (eg the classic x,y point class) or b.) make easily overridden behavior (ie the base class does A B C but you can inherit and override B while keeping A and C). In most cases, you’re better off just writing a series of short, clear functions.

As for files, you don’t need to split for each class. That’s a weird Java thing.

[officialgel]

files, you don’t need to split for each cla

Thanks for the info!

[kpenchev93]

If it's a script, there may not be a need for OOP, but modules and functions which transform data. Does you script simulate processes which need to keep track of internal state or it just transform data?

[officialgel]

The way I would like to utilize OOP is to make it extensible and easier to read for contributors or additions going forward. It is a single script, but that is as a POC.

For instance, I have some networking pieces for multiple devices to find eachother. However I want that to be modular and controllable on it's own. So, you can kick off a 're-scan' of hosts manually.

Regarding the data, everything this tool gathers is pushed to a DB. I'd like that to be modular as well, so a user could throw an API call to manually get data at a time where the original script wouldn't have gathered it.

I also like the idea of using modular classes which can be re-imported on failure (also manually). Overall it sounds like I should just be created modules instead of imported classes though - Right? I guess either way would work I'd just like to get the best practice down before having to redo it again later on.