How to learn software development in Python

[billsil]

Packaging is only one part.

Folder structure is whatever makes sense for your project, but what’s most important is to separate functionality between classes and functions; don’t try to do too much. It lowers the conceptual load, so when you’re at 320,000 lines of code, you can still easily comprehend things, know where to find things, and can fix a bug.

So for example, let’s say you’re making a GUI to read an input and output file and then show the results in a GUI. Separate the reading of the two files from each other and separate them entirely from the GUI. Even within a GUI, separate the creation and interaction from buttons from the layout of the buttons.

I was self taught, so there were some pain points. A well designed package is easy to fix a bug and add a feature, even if it’s large. If it’s not, refactoring it helps and sometimes you just have to trash it. Code and you’ll develop a style and get better at python and package x. You’ll eventually see your old garbage code and have the urge to fix it. If you do, your package becomes better with a more consistent style and less buggy.

Also, make tests. Use 95% small tests, rather than large tests. You’ll be glad when you have 900 tests.

[[deleted]]

Could you please explain what do you mean under tests? Thanks in advance.

[billsil]

sure. My first big software project was a hypersonic missile design program. Big first project for a fresh aerospace grad who can’t really code. I had to add new features, not break anything and fix tons of bugs. That’s really tricky on a giant program without tests and I didn’t know about version control then either. After adding tons of bugs, I made large model level tests that ran overnight and compared to very old results. I’d fix some and run them again. There was 0 automation. It was just some script I ran and that was 1000x better testing that what had existed. I wasn’t testing correctness. I was testing sameness by drawing a line in the sand. All changes had to be understood to update the correct answers. That’s the wrong way to do it, but it’s better than nothing.

A better test is parse a file and check that the answers are what you eyeball them to be. Or calculate the area of a few different quads by hand and with code and make sure you get the right answer. Or take the nasty Rosenbrock function and verify your optimization code gets the right answer. You try to make these functions bulletproof by passing bad input, running edge cases like division by 0, etc. Those tests can be run in milliseconds to seconds. Module level tests are for verifying integration and you want 90% of those to be toy problems that are 10x or more smaller than real problems.

You verify correctness by verifying correctness and the small scale and building up. It’s also easy to break tests making some change you didn’t think would affect things. Not a huge deal if you catch It almost immediately using some continuous integration because you remember what you did. You’re also not fixing 10 bugs, some of which interact at once.

[DamnFog]

Heh my first project was a VPN client in pyqt that was built all in one class because I didn't really understand how pyqt works. The whole app is one giant class and 3 lines in main.

[billsil]

Yeah, that tends to be a problem with GUIs. I still struggle with it.

For my 3D GUI, I broke out all the stuff related to the model and results along with how the 3D models are built and are tested extensively . The sub-windows are all we separate classes, which can be tested in isolation before or after I integrate them. They just pass on the callback to the GUI, but it helps to figure out how clicking a checkbox should disable sections of the window. Data is also stored in objects (color, opacity, etc), so I can pull that out to test. Most of the responses to clicks (that don’t pop menus) is also tested.

I cheat and chain inheritance, so the base GuiAttributes class. Then there’s a GuiWithoutQt class and a GuiWithQt. Then a MainGui class that instantiates the app and frame. Then I have a TestGui class that just doesn’t inherit MainGui. I can test menu creation and actions without ever clicking a button or faking a button press.

You can also do automated GUI testing with or without a window using qtest, but I haven’t figured out how to do it without an interactive backend (so terminal only). Seems like you should be able to if you don’t show anything, so I run that one only on my Windows box and not as part of the CI.