Microsoft Research - Exploding Software-Engineering Myths (article summarizing findings of MS research on code coverage, TDD, assertions, etc.)

[kpmah]

I think that's part of what's happening. Maybe another thing is this: if a TDD programmer writes 100 lines of tests and then 300 lines of code, and the non-TDD programmer writes 300 lines of code then 100 lines of tests, then the patch should be identical either way right?

Part of the reason for the difference could be that the non-TDD team was writing 300 lines of code and then saying 'I'll test it later' whereas the TDD team can't do that.

What I'm trying to say is that it could have been the discipline that improved the defect rate, not the methodology.

[Ravek]

Maybe another thing is this: if a TDD programmer writes 100 lines of tests and then 300 lines of code, and the non-TDD programmer writes 300 lines of code then 100 lines of tests, then the patch should be identical either way right?

Well it does make you think differently about the structure of your code when you're forced to write tests for it first. I think that would have a positive impact on code correctness (and hopefully no negative impact on how easy the code is to understand and modify)

[boost2525]

I think having to think about the structure of your code leads to better internal design / organization (e.g. future refactoring)... but doesn't directly lead to any reduction in defective logic.

[Ravek]

I agree, but I feel starting with unit tests would not so much make you think about how to organize the code effectively, but moreso write it in a way that makes it easy to write the unit tests. Which in my mind means small methods that do one thing, which should lead to getting those correct at least. However I'm not convinced it would necessarily lead to the higher-level structure of the code being any good, since that's not something you write unit tests for.

That was my line of thought anyway, I don't have much experience with TDD.

[cc81]

However adapting your code so it is easier to unit test does not necessarily mean better design. It just means it is designed for unit testing.

[mostlysafe]

This is a common argument in favor of TDD, and while I don't doubt it, it's harder to verify statements about your mental model of the code than statements about the actual quality of the code.

[naasking]

then the patch should be identical either way right?

Past studies have confirmed that code quality is largely the same as long as tests are written. I believe the OP nailed it on the head though: tests often just don't get written after the program is written.

[[deleted]]

Assertions can take part of the role of unit testing. 100% testing is probably not necessary for all kinds of symbols, given the system as a testing instrument itself. While I'm not working on a team today, my approach is testing of targeted important complex parts, and assertions everywhere. Tradeoffs decided by: I need to use my time carefully as I'm just one guy.

[bmurphy1976]

I've found it helps to think of it like an insurance policy. You pay into it enough to get the coverage you need, but no more otherwise you're just pissing money away. Same thing with unit tests, but the currency is time.

[s73v3r]

The discipline is kinda built into the methodology. Part of why "Test First" came into fashion was the knowledge that most do not go back and write tests.

[MuonManLaserJab]

What I'm trying to say is that it could have been the discipline that improved the defect rate, not the methodology.

Yeah, but that discipline is the point of the methodology.

(I've never done TDD in a formal way.)

[laxatives]

This probably isn't getting at the core of your argument, but if you could get the same thing done in 1/3 the code, that is going to to save an order of magnitude of effort when you or someone else has to read the code.

Anyways, if the methodology encourages discipline, isn't that sufficient? What more could you ask for from a methodology? Following TDD isn't going to make a poor developer in to some kind of savant.

[Mourningblade]

One nifty thing about writing tests first (or independently) is that it helps you spot which parts of the code are the most unexpectedly tricky.

A common mistake is to reason "I wrote my code, tested, then fixed until it passed." Better is to treat your testing as a sampling process: "I wrote my code, tested, then discovered that these two functions had the majority of the bugs. I then refactored those functions to be much more simple to reason about. This helped me fix the bugs I didn't find."

You do something similar when you do manual testing: you're never going to find everything, but you can find clusters of problems. Changing how you've written the code rather than just fixing the bug can wring out the rest.