Welcome to

Magenic Technologies Community Blog

Sign in | Join | Help

Aaron's Technology Musings

Who let this guy on the podium?

Does TDD Guarantee Quality? Probably not.

No.  Period.  Full stop.

TDD guarantees quality about as much as religion guarantees morality.  It is probably tempting to draw a cause effect relationship from one to the other, since current TDD practicioners tend to be very good software developers, but the idea that the TDD process invariably leads to quality is, frankly, a case of post hoc ergo propter hoc.  Not too much unlike an assertion I heard many years ago in college that 98% of people who are in prison have drank alcohol - and therefore you should not drink alcohol (leaving out such statements of truth that 100% of people in prison have mothers... which could lead to the false assertion that you should avoid having a mother).

There is a ton of value that TDD brings us.  TDD can:

* Help you design software

* Isolate and crystallize requirements

* Help you avoid writing YANGI code - since code that is not written to turn a test green probably does not need to exist

However, like many other things, it ain't everything.  If you are inventing, say, the next mass produced hydrogen car, would you design each piece by having the mechanical engineer draft the dimensions of each part first?  I am not convinced how TDD can help in the invention process, where you have a vague idea of what the inputs and outputs are going to be for the software component you are writing.

Said another way, during the design process, if I have to figure out a test suite for each iteration of a design that is rapidly changing from one iteration to the next, it can follow that TDD can be a drag on my design velocity.  When I am writing "exploratory code", say, to implement IQueryable over some new thing, frankly, I am going to experiment until I get something very simple working at all.  Same thing applies when I am doing work where I might be applying a somewhat novel algorithm from some researcher to some problem domain where it has not yet been used (sidebar: those, by the way, are REALLY good cases for something like the F# interactive window, but I digress).

Now, for very well understood domains where you are going from a user story on physical medium... a case where you have a pretty good understanding of what the vague shape of the end solution will be, TDD rocks!  But the idea that TDD is a process that can, by nature of the superiority of TDD as a process, produce bug free code, is one hell of a stretch.  That assertion, in my view, relies on the tautology that you write the tests (and this, I will be more broad and say unit, integration, and all other 'coded' and scripted tests that come from developers) - and that quality is measured by the same tests.  In the land of code, it is circular reasoning, and ignores the fact that all creative processes produce something that someone other than the creator will regard as a defect.  This is why even the greatest writers, from Stephen King, to Carl Sagan, to John the Baptist had human editors who inspected the work product, manually, and produced redlines bug reports.  You still need an editor-in-chief.

In other words, TDD is great, but as a process, it does not lead to bug free code.

Update, with example:

Given this:

        [TestMethod]
        public void SillyAssertion()
        {
            specialInt fortyTwo = new specialInt() { Value = 42 };
            Assert.IsTrue(fortyTwo == "Answer to Life, the Universe, and Everything");
        }

 

And this:

    public struct specialInt
    {
        public int Value { get; set; }

        public static bool operator ==(specialInt op1, string op2)
        {
            return true;
        }

        public static bool operator !=(specialInt op1, string op2)
        {
            return false;
        }

        public override int GetHashCode()
        {
            return Value;
        }

        public override bool Equals(object obj)
        {
            return true;
        }
    }

 have we guaranteed bug free software?

Published Sunday, April 20, 2008 11:58 AM by Anonymous

Filed under:

Comments

# re: Does TDD Guarantee Quality? Probably not. @ Sunday, April 20, 2008 8:54 PM

I've never heard anyone in favor of testing state the idea that tests ensure quality. I believe it's a great aid in ensuring quality, but it's not the only way. Where have you heard someone say this?

Jason Bock

# re: Does TDD Guarantee Quality? Probably not. @ Monday, April 21, 2008 7:06 AM

I heard it a couple times during an open spaces talk.  It was an assertion that "quality is baked into the process" or something to that effect.  Whenever I start to hear assertions like that, my bs detectors go up.

Quality is something that people do, not that processes do.  Believe me, you could take the most simplistic, easy to run poke yoke process in the world, and you could find people who could mess it up.

I will give you that the TDD process *can* promote quality given the right people involved, but to say that quality is baked in because you do TDD doesn't follow.

Anonymous

# re: Does TDD Guarantee Quality? Probably not. @ Monday, April 21, 2008 7:45 AM

You are 100% correct.  Testing does NOT ensure quality.  To flip that on its head, not testing can help to ensure the lack of quality.

The quality of your code is only as good as

1) the worst test in your suite

2) the level of your worst (most junior) developer.

One thing you did not mention is that tests help to ease future pain... This alone is worth its weight.

Derik Whittaker

New Comments to this post are disabled
Powered by Community Server, by Telligent Systems