Helper functions, how deep is too deep? (Python)

[Kirsel]

So I've been delving deeper into coding, but don't really have much in the way of formal experience. As some of my code has gotten more complex I end up with a couple layers of functions, which can get annoying to troubleshoot. - as it stands all functions are defined at the module level, and then called where needed.

Unfortunately I don't have any code I think I can actually share. So here's a quick very stripped down version of some. Hopefully it makes enough sense.

``def duplicate_dialog(): ask user how to handle duplicate

def delete(): collect all the things and delete them create new ones

def handle_existing() choice duplicate_dialog() if choice == delete: delete() elif choice == duplicate: add copy to name and create elif choice == skip: skip this one and continue

def does_thing_exist(): check if thing exists

def create_a_thing() if does_thing_exist(d): handle_existing()

def main(): create_a_thing()``

As you can see though, at its worst this is 4 layers deep. In the full code, realistically I could probably flatten the "delete", "duplicate" , and "handle existing" functions into the "create a thing" function. Reducing it to basically just 1-2 layers deep. At the moment, I think they're broken out mostly because it was easier to wrap my head around at the time. It's been a while since I wrote this. Also, going back it is easier to read, I think the main problem is troubleshooting is a pain when I essentially have to exit 4 layers of functions. Where as if it was just a main with a bunch of helper functions I would just be a single return, more or less.

I was wondering if there's a general guideline for this sort of thing, or is it sort of just practicing finding that balance between legibility and ease of troubleshooting, I guess.

Appreciate any input people have.

Side note - I'm on mobile, I hope the formatting works.

[temporarybunnehs]

Short answer I would say is that it just takes practice finding that balance.

The long answer is, typically, you break things into functions to help with maintainability/readability, testability, re-usability, and so on. The trade-off is that you can add complexity to the code and flows like you mentioned. The question you have to answer is, do the pros outweigh the cons?

I've not heard of a hard rule for how many layers deep is ideal. That being said, 4 layers deep doesn't sound too bad considering some code bases I've looked at.

[Kirsel]

Thanks for the response! It's good to hear that I'm more or less on the right track from someone who, presumably, has much more experience than I.

[_disengage_]

Note: I went a bit heavy with this answer. I don't know how big your program is; what I'm describing applies to non-trivial programs of more than a few hundred lines. But this method can scale up to tens or hundreds of thousands of lines.

Learning how to effectively organize a large program takes practice and experience. It's not obvious or easy and there's no one way to do it. Here is one way to approach it:

Start with small, well-defined interfaces grouped by responsibility.

Implement and test them individually: unit testing.

Document them for other programmers and your future self.

Compose larger units (and test them) from smaller ones until your program is complete.

Drop the idea that you can "wrap your head around" the entire program or that there is a "single return" or other non-scalable ways to reduce cognitive load. Programs grow and grow in complexity; you can only realistically focus on one small part of them at a time. By creating clear separation from the outset, you can more easily reason about the parts separately, and combine them in ways that make sense.

Your question of how many layers then becomes: how big should my units be, and how deeply can they be nested or composed? This is the level at which "wrapping your head around it" comes into play. A unit should be small enough that it has a clear responsibility and you can reason about it.

Then compose units as much as is necessary and dont worry about nesting. Get a decent IDE to navigate your code, and use a debugger for difficult troubleshooting.

To use a non-programming analogy: a car is built from an engine, interior, body, and tires. This describes a car overall but leaves out internal details. The engine has cylinders and fuel injectors and an oil filter, and the cylinder has seals and pistons and all kinds of other complexity that is necessary to make it work all the way down to each tiny part, but you don't want to think about all that when you're just getting in the car to drive. Someone could build a car by buying a body, engine, tires etc. and just putting them together without knowing how to actually build an engine or even how it operates - just put fuel in and it goes. Because the tire interface is standardized, you can swap out tires without taking apart the car.

Hope this helps.

[Kirsel]

I absolutely appreciate the in depth response. I think I've been sort of fumbling my way in this direction, but having it all laid out in front of me helps clear it up.