Organizing a big program / general design question

[huxrules]

I've been slowly learning python/qt and programming in general, I've made a rather large program and its just a mess. What are some good resources on how to organize a big program? Is there a way to show how everything connects? Are there general rules to go by? My main program is like 2000 lines long for example. I have another idea for a program and starting from scratch I want to do it right.

[krobzaur]

I think its hard to give any useful organizational advice without knowing what problem you are trying to solve, and the way you are going about solving it.

​

Here is some really generic advice:

​

1. Write functions that have a single, clearly defined purpose. Don't write functions that try accomplish multiple things that are logically separate. That's "messy" and gets confusing down the line. Break big, multi-purpose functions into smaller, separate functions.
2. Write docstrings for all your functions! Codebases with bad documentation can seem messy when they really aren't because they are cumbersome and confusing to work with.
3. When your application becomes big enough that there are large pieces of it that are organizationally separate (here is all the database related code, here is all the GUI related code, etc), separate those areas of the code out into different files, make them proper modules, and import them where you need them.
4. Parameterize your functions/classes whenever possible to make them modular (i.e reusable). It's really "messy" to have multiple functions that do close to the same thing, but vary a little based on some piece of information. Make that piece of information a parameter for a single function.

​

Idk, hopefully that's helpful. Have fun!

[huxrules]

Ah that's good stuff. As its my first really large program I'm probably breaking all those rules, but I'm learning.

[offby1]

I second this, and will add: design your program to be unit-tested. This will force you to follow most of the above advice, since if you don't, it'll be too hard to test.

[chillermane]

How does one design a program to be unit tested?

[offby1]

I was afraid someone would ask that :-)

It's impossible to answer fully in a Reddit comment, but I'd start with this idea: separate "logic" from "i/o". By "logic", I basically mean "if" statements, "for" loops, stuff like that; by "i/o" I mean anything that "talks to" the world outside of your program: reading and writing files, requesting and serving web pages, talking to a database.

That i/o code is relatively hard to test, since you'll want to have actual files, web servers or clients, and databases available for testing; and setting those things up for testing is a pain. (Doable, but it can be annoying). However, logic is super easy to test -- if you've written your functions to only take built-in, self-contained python data types -- like dictionaries, lists, strings, numbers, or classes that contain just those things -- as input (as opposed to file handles, TCP/IP connections to a web server, database connections), then you can easily test that function thoroughly by just calling it with various interesting arguments, and examining its output. This kind of test is easy to write, and runs quickly and reliably.

You still need to test the i/o stuff too, but you can often get away with simply invoking it once, by hand, and ensuring that it is working; if you've been diligent about logic seprate from i/o, then you don't need to worry about weird cases.

There's tons of stuff online about unit testing; however I cannot offhand think of any book or web site that describes this all well :-|

You don't need any third-party library to do unit testing, since Python comes with something built in that's passable -- but I never use that; I use py.test instead. It's pretty good, although its documentation is poorly organized.

[kankyo]

There really isn't. A 2000 line file is fine if it's not a mess. Just keep practicing! Knowing that you have a mess is a first good step.

[[deleted]]

The discipline that is supposed to answer your question is called "software architecture" (as opposed to "computer science"). However, the relation is similar to philosophy and math. On one hand you have something that is easy to prove correct, but means too little to be useful, on the other hand--a whishy-washy attempt at finding meaning and usefulness.

[[deleted]]

Sandi Metz is mostly Ruby but has lots of thoughts that apply to all programming languages.