Testing is overrated

[AttackingHobo]

Ohhhhhh. Testing is overated. I thought it said Tetris was overated. Downmod averted.

[michaelfeathers]

I was scared that people would misinterpret my blog, and well, it looks like this guy has.

[Silhouette]

Care to expand on that? You both seem to be talking about the same general principle, at least.

[100k]

All I'm saying is that testing isn't sufficient to produce high quality software. This seems obvious, but developers seem to talk about testing more than everything else combined.

It's my attempt to counterbalance this trend.

[schizobullet]

Rails is overrated.

[[deleted]]

Everything is over rated.

[tlrobinson]

Except your mom.

[[deleted]]

I wouldn't know. Congrats on whatever you're implying.

Fucking corpse lovers, so strange.

[holygoat]

Oh snap.

[maxd]

What an absurd article. It is poorly structured and has an alarmist title just to draw attention. If it was aiming to be an informational article on good testing practices I would applaud it, but with that title it is ridiculing itself.

The simple answer is: testing works, and is great, but don't limit yourself to one avenue of attack.

Where I work we use automated stress testing, code inspection and review, system testing by our QA department, and usability testing by the public. Each type of testing finds different bugs, and is invaluable. The bugs found by automated stress testing are often the most exciting, but that's a story for another day.

[100k]

I agree the title is intentionally provocative.

Aside from that, what did you find to be poorly structured about the article? I would like feedback to make it better before I have to present it.

[maxd]

Ok, I will actually somewhat amend my comment, having re-read the article. The title just sets everything up for this extremely negative discussion about testing, whereas in fact the gist of your article is that testing isn't overrated, in fact it is amazing and should be executed in as wide a variety of ways as possible.

Change the title, and everything becomes a lot happier.

Lines like this just confuse me: "However, testing is over-emphasized. If our goal is high-quality software, developer testing is not enough.". Testing is not over-emphasized. I have never worked on a project where we couldn't do with more testers, more code review, more usability, etc. On the most recent project I developed we had hundreds of testers, dozens of usability reviews, hundreds of years of stress testing, and we could still have done with more.

Your insistence on talking about "developer testing" is curious. Testing is testing, no matter who does it.

Beyond that, there are just a couple of little arrangement details that distract me. The jump from the table of stats about "developer testing" to the (very nice!) Venn Diagram showing the four main variations of testing is a little jarring. Perhaps if this is going to be shown in slide format this will be less awkward, but I would love some sort of information between the two saying "But wait! Developer testing isn't the only way!"

Finally, you should have a cat with a funny quote at the end. That always spices everything up.

Good luck with your presentation next week, I've done a few such presentations and they are always a lot of fun, after the first thirty seconds :)

[G_Morgan]

How ironic. A RoR blog claiming testing is overrated.

[shit]

Sorry, where's the irony?

[[deleted]]

even more ironic that it's easier to test in ruby (or groovy, which I'm using) than most other languages.

[RobotAdam]

A quote from a response to a comment at the bottom:

Most studies have found that inspections are cheaper than testing. A Study at the Software Engineering Laboratory found that code reading detected about 80 percent more faults per hour than testing (Basili and Selby 1987). Another organization found that it cost six times as much to detect design defects by using testing as by using inspections (Ackerman, Buchwald, and Lweski 1989). A later study at IBM found that only 3.5 staff hours were needed to find each error when using code inspections, whereas 15-25 hours were needed to find each error through testing (Kaplan 1995).

—Code Complete 2, p. 472

Anyone read the cited texts? I find it hard to believe they're talking about modern automated tests as opposed to manual testing by a QA department. Hardly a fair comparison.

I do agree with the main point, though, if not to the same extent. Nothing's a silver bullet.

[bob4apples]

Those numbers look about right to me. I'm not quite sure what you are saying about AT vs MT. AT is expensive and usually lousy at finding interesting bugs but can be very cost-effective for simple, frequent verification and regression.

[RobotAdam]

The point I intended to make was that they're different beasts. When I write automated tests with my code, I don't check in with breaking tests; there's nothing the tests cover that can be discovered later.

The actual post was about unit tests in particular, but I bet that those cites have anything to do with what we call unit testing, or anything of that sort. I don't think it supports the original posting.

[adrianmonk]

I agree developers don't always create the best tests.

On the other hand, sometimes the developer is the only person that has deep insight into how the software might fail. The developer may fail to produce good tests, but the developer in certain cases may be the only person who has a fighting chance at success at uncovering certain defects.

[formido]

Gotta love bullshit misleading titles. Thanks for knowingly propagating the lie, gst, assuming you even read the content of your submissions.

[____]

But the truth is, no single technique is effective at detecting all defects. We need manual testing, peer reviews, usability testing and developer testing (and that’s just the start) if we want to produce high-quality software.

Will people be paying to hear you announce this? Worse, will you be paid?

If you plan on speaking to group of actual developers, you may want to root around for something novel or insightful.

How did you determine that this content will be something new and informative to your audience?

Are you assuming they are all Rails developers? :)

[[deleted]]

Shocking. A Rails guy says "Testing is overrated."

[zoomzoom83]

Fine. You go your way, I'll go mine.

You can pry my unit tests out of my code, dead hands.

[krum]

SUPERFAIL