Test Driven Development

[Papus79]

I know there are some other programmers and software engineers here.

Do you guys use TDD in the 'full prescribed' manner? If not, if not doing it is a matter of economic sanity, what do you think gets the same job done without red-green-refactoring for every line of code?

I ask because we're looking at it right now where I work. While I don't have a problem learning it I don't see it solving the problems we want to solve (ie. fewer hours on a project, fewer hours in maintenance and bug fixes). We're in a position where we've been doing our own projects solo and in that case you get to know them intimately, ie. they become your world for a year or more at a time.

I think our biggest problem is really our clients constantly changing their minds or having to rewrite or update code, based on changes in preference or additional desired features, while it's in production.

For me an example of good self-testing code is having null checks after filtering when you need at least one object (with an error thrown against a try/catch block if it's empty) and similarly qualifying the results of your operations in such a manner where - while it's being built - it can't fail without triggering one of the many data symmetry checks that you might put at the end of it.

I'd love to flag the above in such a manner where I could just have it output red and green as needed because this might sound a bit arrogant but... I don't understand what I'm going to think of in a test-first environment that I wouldn't think of in a code-first environment, at least without having the whole program be a pile of functions mostly fewer than ten lines.

[Steve3007]

No we don't tend to explicitly use TDD. The company for which I work is not primarily a software company. It's an electronic hardware manufacturing company, of which software is a small but significant part. Our techniques for writing software tend to depend on the requirements for the particular project. For example, some of our projects are safety critical and require a high degree of determinacy and reliability in the software. So, for those projects, the requirements and the ability to validate and verify against those requirements come first. In writing a verification matrix, we define, for each requirement and each associated section of code, how compliance to the requirement will be verified. In that sense, those projects are test-driven. In those sorts of projects, since requirements are explicitly agreed with the customer before starting work, any changes are identified and costed. So un-costed "feature creep" is kept to a minimum.

But a lot of the software we write is for internal testing proposes. Typically we write an application in C# for testing a bit of hardware via a serial connection (e.g. RS232). Specification of requirements there is much quicker and more informal.

[Papus79]

That sounds right, and it has to be a plus to have customers sophisticated enough that they understand what it is you're doing.

In the world of hourly billing doing TDD in a dutiful / arbitrary way (the sort of 'I'm a junior developer, don't know what I'm doing so the more CYA the better') can't be financed. I almost get the sense that the community is terrified of explicitly giving developers a tool that just red/green's their own code based on non-intrusive bits that can couple to standard things like ArgumentExceptions with minimal extra detail (like they're trying to bulletproof the profession against people coding at levels that they shouldn't be coding - and it's a zero-sum tradeoff, there's only so much of that which you can do before everyone else is losing a lot of time and profitability).

[Pattern-chaser]

Welcome again to the Online Philosophy Club's Programmer's Forum!

If not doing it is a matter of economic sanity, what do you think gets the same job done without red-green-refactoring for every line of code?

I have had this discussion a number of times. In short, months of debugging can save hours of test-writing!

[Pattern-chaser]

The team discussed in this case study realized a 20.9% decrease in test defects in V2 in which they wrote automated unit tests, relative to V1 which did not have automated unit tests. From V1 to V2, product quality improved which the team attributes to the introduction of automated unit tests into their process.

A comparison of case studies of TDD teams, as reported in Section 2, indicates that additional quality improvements may be gained by writing unit tests more incrementally as is done with TDD. The TDD teams had 62% to 91% fewer defects. The incremental nature of writing unit tests via TDD may cause teams to write more tests. The TDD teams had a higher test LOC to source LOC ratio and higher test coverage. These results are consistent with a controlled case study conducted in Finland of three nine-week projects [20]. Results of this case study indicated that test coverage was higher when tests were written incrementally before writing code.

Link to article

The impression that TDD is an indulgence we can set aside for time-critical projects is wrong. It is the very opposite of the truth. TDD saves development time (if we include debugging in 'development time').

TDD doubles coding time. But debugging takes anything up to 80% of the project's overall development time, and debugging time is drastically reduced by TDD. The overall result is faster delivery of a product that contains a LOT less bugs.

[Pattern-chaser]

I think our biggest problem is really our clients constantly changing their minds or having to rewrite or update code, based on changes in preference or additional desired features, while it's in production.

Changing requirements are a fact of life. If your development process relies on frozen requirements, you will always fail. Requirement changes are a pain in the ar$e, but they are part of the real world, and we, as developers, have to find ways to cope with them. Does our development process allow for continual requirement changes? If not, why not? Do we develop in ivory towers, or in the real world? Is our development process based in those ivory towers, or in the real world? And so on...

[Papus79]

TDD's one of several types of regression testing. I'm interested in setting something up, just trying to figure out what matches our use case. This is why I'm not wild about the Bob Martin 'TDD or flip burgers', if you're getting reliably self-testing code as an end result and you're really not getting that many errors so much as wanting to repot / update projects as part of a maintenance plan and want the red / green to be able to tell if the version updates broke anything. I don't know that there's specifically any 'magic' in one way or another if what you're doing delivers at the end of the day.

The other thing I might add - the real PIA with most of my programming isn't back-end stuff, it's UI, and particularly the length of builds as I'm testing something to figure out why it's not behaving as expected. If it's an Angular project where you're rebuilding node_module and trying to red/green/refactor that part - you'd be unemployable. Unless you're heavily invested in AWS (Amazon Web Services) for your processing power those builds can easily take 3 to 5 minutes per change.

I don't think anything is a panacea (I've found Thomas Sowell's 'No solutions, only tradeoffs' to be accurate much more often), I haven't seen panacea anywhere in my life and listening to DHH, Martin Fowler, and Kent Beck discuss things it sounds like they'd agree that there are no panaceas in the coding world either. Certain techniques work really well for certain use cases, and that's what I'm trying to get my head around.

[Pattern-chaser]

TDD's one of several types of regression testing.

I don't think so. TDD is a coding/design technique, not a testing technique, odd though that may sound.

[Pattern-chaser]

The TDD teams had 62% to 91% fewer defects.

The impression that TDD is an indulgence we can set aside for time-critical projects is wrong. It is the very opposite of the truth. TDD saves development time (if we include debugging in 'development time').

TDD doubles coding time. But debugging takes anything up to 80% of the project's overall development time, and debugging time is drastically reduced by TDD. The overall result is faster delivery of a product that contains a LOT less bugs.

When my team adopted TDD, we saw a decrease of 80% or more in bug reports. The bugs we did find were nearly all down to one team member who worked remotely from the rest of us, and only pretended to do TDD... Ironically, when we were all later made redundant, this team member thanked me on social media for teaching him to use unit-tests, because it made him more employable, and secured for him a new job!

[Papus79]

When my team adopted TDD, we saw a decrease of 80% or more in bug reports. The bugs we did find were nearly all down to one team member who worked remotely from the rest of us, and only pretended to do TDD... Ironically, when we were all later made redundant, this team member thanked me on social media for teaching him to use unit-tests, because it made him more employable, and secured for him a new job!

How did he 'pretend' to do TDD without even knowing unit tests? You don't need TDD to do unit tests but if you aren't doing unit tests you aren't doing TDD.

[Pattern-chaser]

When my team adopted TDD, we saw a decrease of 80% or more in bug reports. The bugs we did find were nearly all down to one team member who worked remotely from the rest of us, and only pretended to do TDD... Ironically, when we were all later made redundant, this team member thanked me on social media for teaching him to use unit-tests, because it made him more employable, and secured for him a new job!

How did he 'pretend' to do TDD without even knowing unit tests? You don't need TDD to do unit tests but if you aren't doing unit tests you aren't doing TDD.

He wrote some fairly poor 'unit' tests after he'd coded and 'tested' his code by running it on the hardware. He coded only for the wage; he had no interest in our profession, or in getting better at it. He wrote tests because we forced him to, but he wouldn't do TDD... I failed to convince him of the efficacy of TDD.

[Papus79]

I'm still not understanding how he got away with not writing unit tests and passing of what he wrote as unit tests. They're extremely simple, small, binary things. It's like checking to see if a light switch works except that you can't simply say you did - you either have code that does it or you don't, you can't really bluff it.

[Pattern-chaser]

I'm still not understanding how he got away with not writing unit tests and passing of what he wrote as unit tests.

First he wrote the code as he always had, no planning, just sit down and code it. Then he 'tested' it as he always had, by running the program on an emulator, and observing it working. Then, because he had to, he wrote the 'unit-tests', as few as possible, great long sprawling tests that each tested dozens of things, not just one. Then he checked in the code, none of which we had a sight of until that point, and we saw the tests. By then, it was too late in the project - projects are always in a rush - and no corrective action was possible. In other (employment) circumstances, maybe in another company, he would've been sacked.