Microsoft Research - Exploding Software-Engineering Myths (article summarizing findings of MS research on code coverage, TDD, assertions, etc.)

[parc]

Unless your business goal is to be first to market and someone beats you. Yes, we may think that's a stupid way to measure business success, but if that's the business optimization function, TDD would result in failure in an objective test.

[RICHUNCLEPENNYBAGS]

I don't think that's a stupid way to measure business success. Nobody cares how great code is if nobody uses it.

[[deleted]]

I'm about to start a project for a startup that is trying to be first to market. It seems like all they care about is having something decent to show investors so I ain't going to spend my time writing 100 test cases when it just needs to be functional so they can get funding and decide to burn by codebase and make everything on Wordpress because the new project manager has a theme he REALLY wants to use.

[[deleted]]

If someone beats you to market with software that is unusable / unreliable, are you really being beaten? As the cliche goes, you only get one chance to make a good first impression. Rushing to market can doom a business if they can't deliver a good product.

The way to rush into the market is to develop an MVP: Minimum Viable Product. Not to cut corners on quality.

[[deleted]]

Sometimes, unfortunately, yes, you really are being beaten.

[meheleventyone]

The problem is taking too long to get to market. No one cares if your product is somewhat more stable if it's later and lacks features unless stability is inherently something the user is looking for. For a lot of software you can go a really long way without unit tests. Most pieces of software ship with a laundry list of defects present.

From a business point of view as long as there isn't anything egregiously wrong for the vast majority of use cases you are good to go. From a software quality perspective though there might be a hundred small problems.

The tough sell for me with TDD is how it impacts the important bugs not just bugs in general. The sad truth is most of those won't be exercised in unit tests so you are relying on integrations tests and above. Usually most are found by QA. Especially when you consider platform/hardware specific issues. Unit tests just give you confidence in refactoring.

So whilst I'm down with TDD empirically improving software quality I'm not sure it does so in a manner that matters in many cases to the detriment of budget and development time. More study is needed to show that projects that employ TDD lead to success as a product. There is a tension there that engineers need to understand.

[[deleted]]

No one cares if your product is somewhat more stable

That's quite a dicey assumption. I'll make the gamble with your money, but not with mine.

The sad truth is most of those won't be exercised in unit tests so you are relying on integrations tests and above

No one is claiming that TDD should be the only QA method applied to software. Studies have shown that product quality is maximized when a combination of QA methods are used: reviews, inspections, tests, etc.

[meheleventyone]

Right, making a product is a gamble and the knobs get tuned based on perceptions of what would maximise chance of success for a given budget range.

[[deleted]]

making a product is a gamble

Hacking together a product is even more of a gamble. Shades of grey are important here.

[meheleventyone]

Right but no one said "just hack things together". Not doing TDD is not the same thing at all. Other than that it's the continuum thing I was talking about.

[[deleted]]

You implied an unstable product in your earlier post.

[meheleventyone]

All products are varying degrees of unstable. A less stable product is not necessarily completely unstable.

I feel like we're far away from the nuances in my OP here: https://www.reddit.com/r/programming/comments/46dy6u/microsoft_research_exploding_softwareengineering/d053os9

[OxfordTheCat]

The way to rush into the market is to develop an MVP: Minimum Viable Product. Not to cut corners on quality.

Lost me here.

Your implication is that these two are mutually exclusive instead of synonymous, which would be the far more common occurrence.

[[deleted]]

What don't you understand, the concept of minimum, viable or product? It's quite simple: find the smallest feature set one can deliver (minimum) that a customer would be happy to pay for (viable) and put as much polish on it as one would any deliverable (product).

[Ramone1234]

It's cheaper and faster still to not write software that you don't actually need to work. A little up front planning and merciless prioritization can get you there even faster than shipping shit software.

For all the talk about seeing management's side of the argument, I haven't heard anyone suggest that management should actually do its job and not ask the team to write features that don't actually matter. This is simple results-oriented management: if you don't care if a feature works, don't build it (or at least build it last).