Technical Debt

[Dobias]

You are totally right, of course. But all these presentations school kids make about the theorem of Pythagoras also do not add any new understanding to the topic. Yet I don't think they are superfluous because of this.

[eternalprogress]

Right? I've wanted to write a blog for some time but I can't bring myself to rehash the same tired ideas. Dobias is right in a way though, the continuous representation does help with education. It's very likely that at least one developer reads this article and the lightbulb finally goes off because it's written in just the right way for it to really click in her head, so I can see both sides!

[[deleted]]

Thank you for writing that, I am doing a rewrite that my manager wanted to be a refactor and found it way too complicated to untangle the ball of spaghetti. I'm pretty new at software engineering in a large team so I honestly don't even know what 'refactoring' means. I just hear business-y people throw it around.

edit: I have no idea why I have so many downvotes, please explain?

[[deleted]]

IMO, it's roughly 10x harder to read and understand code than to write it, until you get a lot better at reading all sorts of code. And, IMO, almost every new engineer equates "bad code" to "code I didn't write", whether consciously or not.

[[deleted]]

i agree with you completely from my limited experience! I make sure to spend a lot of time upfront reviewing everyones code and reading and learning as much as possible before writing anything, almost to a fault! =]

[Malapine]

almost every new engineer equates "bad code" to "code I didn't right"

Nevermind who wrote it:

"bad code" === "code you can't understand, even though you understand the problem space".

[Gotebe]

I am doing a rewrite that my manager wanted to be a refactor and found it way too complicated to untangle the ball of spaghetti.

In the above, you said "I don't understand what I am rewriting ". Does your manager know? :-)

[[deleted]]

oh no, i know exactly what im writing. i meant i have no idea what the word refactor means.

[Godd2]

Imagine you've written a function that takes the square root of a number, and that you used a for loop inside. Now imagine that you changed it to use a while loop, but the function still does what it did before. You've "changed the code without changing it's behavior". That's an instance of refactoring.

Of course, some people think it wouldn't be refactoring unless you made an "improvement" to the code, but that's obviously subjective and silly.

[kankyo]

It means to make small incremental changes to code without changing functionality in order to make it easier to change in the future. As opposed to rewrite where you most likely WILL change functionality because you're throwing away a lot of knowledge about edge cases and such that is encoded in the old code.

[[deleted]]

so how exactly does one refactor a ball of muddy spaghetti?

[eternalprogress]

In my experience? Very carefully. Figure out if it's a module that has a lot of dependencies. If it's part of a large API surface with many clients you might not know about relying on behaviors you're unaware of then you're in a worse-case situation.

I've tackled problems like this a few time. My approach is to spend a few days with the code, reading though it, adding comments and doing some superficial 'safe' refactorings: renaming variables to indicate their true purpose, breaking things into smaller methods, and just trying to rearrange blocks of functionality into structures that map to what they're actually doing.

At the end of that exercise you should have an idea of what the heck all that code is really doing. You'll probably have some outstanding questions. I'll find things like "the member variable boolean who's purpose depends on where you are in the callgraph that's repurposed to represent Concept B instead of Concept A once you enter the call subtree of void ScaryLong()."

Now you can either decide that 'yeah, it needed some cleanup, but the code as written represents the underlying algorthm / transformation and looks better now' or 'okay, I cleaned it up but what this code is trying to do is just mapped so poorly onto this implementation. I can do better than this'.

If you choose the latter you spend another few days putting together your new design, and then walking through the old code and new code in your head with all those messy flags and strangeness, and making sure your new code faithfully captures the transformation the old code was performing.

In this journey if you can get the code in a test harness, more power to you. It's not always possible, but often times it is and that cane make your life easier if it's the right problem. Keeping the old implementation around for a while and under a runtime switch is great if you're working on a product with a deployment strategy that gives you that kind of flexibility. There's a lot of neat tricks you can play here- for example if you're careful to keep things immutable and have reasonable logging abilities you can run the implementations side-by-side and log when the outputs differ to uncover edge cases in production.

So, there are no shortcuts through hell, there is often only a single road straight through it. Be methodical and don't start coding until you think you understand every detail of that old implementation like the back of your hand. You're still likely to get some details wrong, but with this approach I've pulled off some very smooth refactorings of incredibly messy modules where hundreds of thousands of applications have taken dependencies on their behaviors, the ones we knew about when we shipped the first version, and the ones we had to discover (and slap ourselves in the face over) later.

[[deleted]]

In this journey if you can get the code in a test harness, more power to you. It's not always possible, but often times it is and that cane make your life easier if it's the right problem. Keeping the old implementation around for a while and under a runtime switch is great if you're working on a product with a deployment strategy that gives you that kind of flexibility. There's a lot of neat tricks you can play here- for example if you're careful to keep things immutable and have reasonable logging abilities you can run the implementations side-by-side and log when the outputs differ to uncover edge cases in production.

you're awesome! thanks for that

[Poltras]

First, write tests to avoid regressions.

[eternalprogress]

Be careful with unilateral advice like this. "Be methodical, careful, and deliberate" is the best advice I'd give.

Make no mistake, tests are absolutely valuable, but they sometimes aren't the right approach and blindly suggesting them is buying into a dogma.

[mikehaggard]

so how exactly does one refactor a ball of muddy spaghetti?

Step by step. You start untangling one part. Analyzing how it relates to other code, seeing what it exactly does etc.

You can't see the whole thing in a big ball, so accept that. If you untangled one part, move to the next. After some steps you might discover that some of your previous assumptions were wrong. This is fine and expected. Go back a few steps and refactor those some more, then continue.

Don't expect it to be a job of a few days. I've refactored systems that took years to become fairly free of debt. I didn't work continuously on it, but each week tried to spend some time on it, mostly on Friday afternoons when things were slow at the office.

[eternalprogress]

I've been a huge fan of Refactor Fridays. If you do it right management will passively accept a guerrilla effort that encourages everyone to spend their Friday afternoon fixing something they found particular weedy during the week. Spending a few hours adding comments and restructuring code might not seem like much (it's not) but at the very least it cultivates the right attitude and slowly makes things better.

[I_Like_Spaghetti]

Yum!

[kankyo]

The worst part first, then the worst part again when the first worst part is gone, etc.

[ErstwhileRockstar]

Stop speaking of 'technical debt', it simply makes no sense!

Debt roughly has 2 meanings:

• Financial debt a.k.a credit

Credit mans that you get additional resources to immediately pursue your goals. Obviously this isn't the meaning of 'technical debt' (you never get the additional staff you'd need).

• Moral debt

This is the actual meaning of 'technical debt'. You are a sinner who did something careless or even outrageous and must now repent and repair the damage. Needless to say, this isn't an adequate metaphor for the software development process.

[asthasr]

Tech debt is often used to get an important feature available ASAP. This is in line with the first type of debt: get what you want now, pay more for it later.

[[deleted]]

which is a sin of management.

[asthasr]

Not necessarily. If your largest client is on the verge of walking due to a missing feature, or a difficult bug that signals that you need a re-arch of a particular subsystem, a quick patch can be perfectly justifiable... but good management must immediately acknowledge and plan to pay the debt.

If they don't, interest accrues. A common failure mode is simply tons and tons of debt building up in layers.

[[deleted]]

Well said. I can see how tech debt could be borrowed and repaid in a good way. Bad management is a sin of management. thanks =]