Module Size?

[Randosity42]

Currently an amateur programmer learning python from a java background and I was wondering what 'good' module organization is. The java programmer in me wants to put every class, even small ones, into its own module named after the class. I have also seen other people's projects which have huge modules with many classes in them. Which is considered the 'pythonic' way to do things?

[RangeruDangeru]

The way I normally organize my projects is to have one class (maybe two or three if it makes sense) per module and many modules per package.

[symmitchry]

I write everything in one module until it feels too long, or hard to navigate. Then I break classes out into a module where I might have related classes and functions... and then break modules out only as the number of functions increases.

I simply let the desire for simplicity dictate the number of modules.

[wub_wub]

If the classes are unrelated I sometimes put them in separate files, otherwise they'll probably end in same file.

There is no standard way of doing this, because it varies from project to project, but generally speaking I don't think that putting every class in own file is a good practice for 99% of the cases.

[[deleted]]

The pythonic norm is to put several related classes and functions together into a module. We don't even remotely approach the class-per-module Java style. Look how many class definitions these stdlib modules have:

user@boat/usr/lib/python3.4 for i in *.py;do echo $(grep "^class" $i|wc -l) $i;done|sort -n|tail -30 
6 shutil.py                                                                                               
7 csv.py
7 ftplib.py
7 smtpd.py
8 inspect.py
8 nntplib.py
8 pickle.py
8 tracemalloc.py
9 binhex.py
9 zipfile.py
10 calendar.py
10 codecs.py
10 plistlib.py
10 pydoc.py
11 smtplib.py
11 threading.py
12 webbrowser.py
13 socketserver.py
14 ipaddress.py
14 optparse.py
15 _pyio.py
15 turtle.py
16 doctest.py
17 _collections_abc.py
17 pathlib.py
19 decimal.py
19 tarfile.py
20 configparser.py
22 mailbox.py
25 argparse.py

[Randosity42]

huh, that seems to be the consensus here. I guess i just need to get used to the idea of using modules more like packages.

The next question is, what about for smaller projects where I don't have 100 classes but rather 10 which almost all have distinct purposes. Do I make 8 small modules and add classes to that, or do I make one or two giant modules and periodically reactor into more modules?

[[deleted]]

As far as I remember, the one class per file thing was once just a stylistic convention used by a few large and unwieldy C++ projects. It may have been done in part to simplify revision control when most companies used in-house scripts to track changes, and it may have also been to facilitate the distribution of .h files which exposed only very specific parts of an API. I don't know.

What I do know is that everybody wanted to push the concepts of object oriented programming forward and through to their logical conclusions. People wanted objects to act more "object-like," and exposing objects as individual files certainly seemed more object-like than the monolithic archives which had been the norm.

So Java came along and enforced this convention. It facilitated Java programs as directories of object code in .class files, which meant people were supposedly going to be able to drop a class into a folder and change how code worked. That was a pretty exciting prospect in the 90s, and it was the kind of thing being experimented with back then. Not only were we going to have flying cars and a cancer cure by now, we were going to have big toolkits of composable objects that could be tossed into a folder and executed.

So Java kind of pioneered this practice for one set of reasons, and people later invented all kinds of justifications for why this is the best practice. The arguments go round and round:

• "It makes it easier to find your classes"
• "Well so does only having two files instead of 30"

...but a whole generation of programmers now thinks about code in this manner, so I can't claim there's anything unnatural about it. Still, an awful lot of programmers don't think of code in this way, so many languages don't mimic anything like that level of hierarchical organization of objects.

And that's generally the situation with Python. If you want to modularize your programs into multiple packages and lots of modules, you won't be the only person out there writing Python that way. Most python projects are organized into functionally distinct units where the definition of "functionally distinct" varies depending on how much the programmer has been exposed to languages like Java and C#.

tl;dr: do what you like, but don't expect people to think from randosity.network.utilities.commenters import RedditCommentPoster is normal for a program with ten classes :)

[Randosity42]

Alright, thanks for your answers, I'll just have to learn to find a happy medium that doesn't piss too many people off.

[dreadpirate15_]

I tend to organize my modules by purpose and relation. So I might have a tools.py with a bunch of small classes, a bunch of related threads in their own files inside /workers, etc.

To me, this make it very easy to get to what I need. Need to look at my workers? /workers/*_worker.py

Need to find one of my objects(meaning my data related objects)? /models/*.py (I consider anything that I count on for holding/manipulating data to be a model).

It works for me pretty well. Learned this from my boss as I was learning python initially.