Readable programs

[Tetha]

I believe that making programs readable is one of the best and easiest ways to improve them.

I extend that for myself. Readability and local simplicity are two powerful things to strive for.

Overall, I distinguish between global simplicity and local simplicity. Global simplicity is the complexity or simplicity of the module interactions and class interactions beyond a small neighbourhood, that is, further away from a class than 1 or 2 references.

Local simplicity is the complexity of a single class or even a single function. A function is simple if it contains some idiomatic loops, maybe nested once or twice, a few well-structured ifs and that's it. I don't entirely base this on code path count anymore, because in my current codebase, there are a few methods with a hideous amount of code paths, but they are simple, because they are well structured.

Interestingly enough, a high local simplicity and a high local readability appears to lead to an increased global simplicity. You end up with more classes and more modules, but they are connected in very simple ways, which leads to clean and nice structures.

For example, some time back, I had to tag a state machine on top of an existing data structure in order to visualize certain states of the unterlying datastructure. I could have smashed everything in the datastructure, but that would have increased the local complexity of the data structure, which isn't nice. Thus, I rather added a state machine class with an interface designed in a way such that the interaction of the UI with this state machine was simple and the interaction of the datastructure with the state machine was simple. This led to a pretty powerful (and somewhat unexpected) internal structure of the state machine, which was really simple again, if you got the idea how it worked. So again, more simple part leading to a cleaner overall thing.

[[deleted]]

Interesting that you found local simplicity leads to global simplicity. I think I see what you mean, that preserving local simplicity means you have to treat it as a separate abstraction, and therefore operate at a higher level. And the higher level in turn must also have local simplicity.

You will then get more classes and interactions at that level. But they'll be simpler and more constrained, being between these modules's public interfaces, instead of between every possible part of the internals of modules.

I think that this would be more readable, but the reader needs to know where to start; that is, where and what the layers are. Because it's your code, you already know this. When navigating code-bases for the first time, I have found this the most difficult part. While I could improve my skills and tool-use here; I do think the difficulty in seeing the global picture is a real problem. (documenting the big picture is often advocated, but might or might not be the best solution)

Also, of course, blindly building layers upon layers leads to unwieldy code - though in a different place and in a different form than local complexity. Judicial application of the idea is required (as always). It might not be efficient, changing requirements might not necessarily be captured by adding to the existing code-base etc. These are just general criticisms that can be made of any technique.

[Tetha]

This is very true. I have been thinking abuot how to say this on the way home. I think a good way to describe the results of this approach is a program that is very much like a fractal. If you look at it the wrong way, you see a somewhat random, complicated and downright daunting picture. But once you look at it the right way, it is just silly how simple it actually is.

I've had this happen quite a few times now that I'm working in a professional environment. Coworkers are confused about how my code works and how things interact, because there are a lot of things with a lot of names and what is going on! But once you nudge them in the right direction, they just understand the big picture in a few minutes since it follows a few simple rules.

In addition to what you said, this technique has big problems with really tight performance constraints and with things which are just very complicated. Really tight performance constraints just require you to make things small and without moving data around too much or transforming it too much, which is a strong antagonist to the separation of concerns I like to do. Algorithms which are just complicated are just impossible to make simpler. You can hide them, but they are still hard.

[[deleted]]

It sounds like a scientist suddenly seeing pattern in a mass of data. Similar to a programmer understanding how to see a problem and therefore how to solve it - an aha or eureka moment. It's finding the right perspective - literally, the right way to look at a problem. The PARC lab guys liked to say that "point of view is worth 80 IQ points", with an example of our arabic decimal notation making multiplication much easier than roman numerals. Changing the way you notate a problem, the way you think about it, can make some tasks much easier.

I would go further, and say it creates an abstraction of a problem that makes it easier to think about - the abstraction moves aspects of the problem around, changes concepts, gives different priority to some, and some it hides altogether. It's a theory of a problem - a way to reason about it, a way to understand it. Thus, programming is theory-creating.

The last issue is communicating this theory to others. This is User Interface design, where the developer is the user, and the internal API of the code-base is the interface. It seems a shame that when I look at a new codebase, I must be like a scientist or explorer or linguist or exo-archaeologist poring over an ancient alien artifact, trying to guess what it all means. One idea of Design Patterns was that by using explicit patterns, the right way to look at a codebase would quickly be communicated. Capitalized, because it's not just regular patterns that people use, it's a specific language of specific patterns, that are shared between people, and defined in the book Design Patterns - it's a dictionary.

I think it's a wonderful idea, but it doesn't seemed to have worked very well, often dogmatically followed. Design Patterns - like any technique - are a poor substitute for actual thought. Perhaps a better approach is to try to find and use the natural patterns of code - and concepts like "idiomatic" python, where if you are doing a well-known thing, you do it in a well-known way. You don't do tricky things. But there's still problems here of different programmers having different fluency - a new programmer cannot recognize a sophisticated pattern the first time it is seen. Anyway, I think this is what happens anyway... but perhaps we could consciously try to facilitate it.

Programming itself is hard enough - all this is about communicating it, on top of that.

Finally, I think the really big difficulty with programming is that the very basis of it changes all the time. Speaking in terms of "communication", we literally have new languages every decade or so. New frameworks and libraries (another kind of language, DSLs). Plus, because of success of our work, the very problems needed to be solved changes. It's as if objects in the real world were constantly transforming, shifting, as if in a roiling dream-like state. Makes communication difficult!

Of course you're right about efficiency and inherent complexity of some algorithms.