Tech Book Face Off: Growing Object-Oriented Software, Guided by Tests Vs. Agile Testing

It's been nearly eight months since I've done a Tech Book Face Off. That is way too long. Granted, I had been reading plenty of other stuff—math books, physics books, and econ books—but it's high time that I got back to some software development books. One area where I thought there was still somewhat of a hole in my understanding was automated testing, so I picked out a couple books on testing that I thought would be good: Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce and Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin and Janet Gregory.

VS.

I'm not going to lie. Getting through these books was kind of a slog. I had hoped to learn lots of great new insights on how to go about automating tests and working testing into the software development process without going crazy. I found out that pretty much everything I needed to know was already covered in other books I had read, especially Clean Code and Agile Principles, Patterns, and Practices in C#, two books I've reviewed previously.

While I have some misgivings about panning anyone's work because these authors obviously put tons of time and effort into their books, I don't want to mislead any readers about the value that I got out of these books. After all, these reviews are supposed to give my assessment of whether a pair of books is helpful or not. Sometimes both books are good and complement each other nicely, sometimes one is clearly better than the other, and sometimes I don't find either one very useful. I try to describe those opinions as respectfully and honestly as I can while providing some of the reasoning behind those opinions. With that in mind, let's take a look at these two books on agile testing.

Growing Object-Oriented Software, Guided by Tests

The premise of this book is something I totally agree with. In many ways software grows and evolves over time. Freeman and Pryce nicely described three distinct ways that a code base grows and changes through refactoring:

Breaking out: When we find that the code in an object is becoming complex, that’s often a sign that it’s implementing multiple concerns and that we can break out coherent units of behavior into helper types. … Budding off: When we want to mark a new domain concept in the code, we often introduce a placeholder type that wraps a single field, or maybe has no fields at all. As the code grows, we fill in more detail in the new type by adding fields and methods. With each type that we add, we’re raising the level of abstraction of the code. Bundling up: When we notice that a group of values are always used together, we take that as a suggestion that there’s a missing construct. A first step might be to create a new type with fixed public fields—just giving the group a name highlights the missing concept. Later we can migrate behavior to the new type, which might eventually allow us to hide its fields behind a clean interface, satisfying the “composite simpler than the sum of its parts” rule.

I find that a lot of my software design progresses in these ways. The architecture of a system emerges over many iterations, step by step. This constant flux is a characteristic of Agile development in particular, and it can be an issue for teams and management that are new to it. The authors make some good points about that initial discomfort as well:

[I]ncremental development can be disconcerting for teams and management who aren’t used to it because it front-loads the stress in a project. Projects with late integration start calmly but generally turn difficult towards the end as the team tries to pull the system together for the first time. Late integration is unpredictable because the team has to assemble a great many moving parts with limited time and budget to fix any failures. The result is that experienced stakeholders react badly to the instability at the start of an incremental project because they expect that the end of the project will be much worse.

The goal with Agile development is to not end up in crunch time at the end of the project, but instead spread the chaotic stuff more evenly over the full length of the project, making it more consistent and manageable in the long run.

The first part of the book was much like these excerpts with clear, well thought out discussions of TDD (Test-Driven Development). The second part of the book was an extended example of developing an internet auction bidding program with Agile testing principles, and that is what I had a hard time with. It became increasingly difficult to stay interested as the example went on.




There is something to be said for working through a non-trivial example to better show how a process would work in a more realistic project, but the example that they used seemed too complicated to me. Maybe it was because I haven't written in Java since college and I'm not familiar with any Java frameworks. However, I hadn't done much programming in C# before reading Agile Principles, Patterns, and Practices in C#, and I didn't have any problems with the examples used in that book. In this Java example, the authors seemed to pick a bunch of graphics and networking frameworks that required some prior familiarity to understand what was really going on. For all I know, they are common Java frameworks, but using them served to narrow the audience that could follow along. The complexity of the example and the prior knowledge required obscured the message that the authors were trying to get across.

Once the complexity of the application increased, the big ideas tended to get buried in the details of working through the code. The main point that the authors kept coming back to was that encapsulating related functionality enabled programmers to understand the code quickly and change it easily in one place. This is programming nirvana, but it was hard to see how that encapsulation developed through all of the details of their code samples.

Another issue that came up was that after a couple of tasks were completed, it wasn't clear that it was useful to continue with the example. New concepts and benefits of TDD were no longer being demonstrated; we were just plowing through the rest of the example using the same process we already know about. The danger with a long example is that the focus changes from teaching concepts using a demonstration to completing the example for its own sake. It's a difficult balance that I'm well aware of. You start off with good intentions, but as you get further into the example, it takes on a life of its own. After a while you pull back and wonder how you got where you are, and are you really saying what you mean to say anymore. The overarching ideas have gotten lost in the details of the example. In this case, the goal should have been to show how software can grow efficiently while using TDD, not the gritty details of how to implement an auction sniper.

Even though the extended example was a drag, the authors still had some nice insights into practical testing:

Some TDD practitioners suggest that each test should only contain one expectation or assertion. This is useful as a training rule when learning TDD, to avoid asserting everything the developer can think of, but we don’t find it practical. A better rule is to think of one coherent feature per test, which might be represented by up to a handful of assertions. If a single test seems to be making assertions about different features of a target object, it might be worth splitting up.

I like the pragmatic nature of this recommendation. Dogmatically splitting up tests so that each has a single assertion only serves to extend your test time and bloat your test code. When a set of assertions focus on one feature and have the exact same setup and tear down, they should be combined into one test.

Overall, Growing Object-Oriented Software, Guided by Tests was okay, but I can't recommend it. There are much better books out there, and this one didn't add much to my understanding of TDD. If you're an experienced Java programmer that happens to work with OpenFire, Smack, Swing, and the other libraries that they used in their example, you may get more out of it. Otherwise, I would pass this one by.

Agile Testing: A Practical Guide for Testers and Agile Teams

After getting through the last book, I was hoping for something a little more engaging with Agile Testing. This book was written by a couple of testers who have worked on a number of Agile teams and gave their perspective on how testers could fit into the Agile process after the programmers on the team have adopted TDD. Crispin and Gregory make the case that there are many other types of testing that still have a place in improving software quality, and I strongly agree. Automation with unit and functional testing only scratches the surface of the testing that needs to be done for a product, and having dedicated testers on the team that offer their unique perspective and talents is invaluable.

I did find some insights that resonated well:

If you’ve worked in the software industry long, you’ve probably had the opportunity to feel like Lisa’s friend. Working harder and longer doesn’t help when your task is impossible to achieve. Agile development acknowledges the reality that we only have so many good productive hours in a day or week, and that we can’t plan away the inevitability of change.

This is, of course, one of the key arguments for adopting an Agile process, and you'll see similar statements in any Agile book you read. Most of the book was much like this quote—reasonable and well thought-out—when taken in isolation, but taken as a whole, it was excessively verbose and redundant. The authors were constantly discussing the benefits of automated testing, the use of this or that tool during the testing process, or how you really don't want to make an iteration a mini-waterfall process. I felt like they could have easily covered the same amount of material in less than half of the 576 pages they took to do it.

The book was organized into five parts:

• Introduction to Agile Testing
• Organizational Challenges in moving to Agile Testing
• The Agile Testing Quadrants and the different types of testing that makes up each quadrant
• Automation of any testing that warrants it
• An Iteration in the Life of a Tester and how testers fit into an Agile development environment

At a high level it seems well organized, but there was so much overlap between these sections that much of every chapter was repetition and cross referencing. After a while the periodic tidbits of wisdom didn't seem worth the trouble of wading through pages and pages of generalities. Even the beginning of each chapter had an extra repetition of the rest of the chapter with a mind map that exactly duplicated the chapter's section headings. Maybe that's useful to some, but it just added pages to the book for me.

Some of their points were surprising as well. For example:

“AADD,” Agile Attention Deficit Disorder. Anything not learned quickly might be deemed useless. Agile team members look for return on investment, and if they don’t see it quickly, they move on. This isn’t a negative characteristic when you’re delivering production-ready software every two weeks or even more often.

I wrote one small note on my Kindle when I highlighted this quote, "definitely sounds like a flaw to me." This attitude struck me as having a cavalier disregard for deeper thought and more exploratory learning, especially for difficult problems. Maybe it works when you're implementing yet another CRUD app, but some software requires careful research and design to make something that's going to have a big impact. My interpretation of this statement is that I shouldn't be doing Agile Testing if I'm developing ground-breaking software, but I don't think that's really true. I think Agile promotes examining the trade-offs before making a decision on how to proceed, and sometimes the right decision is research and learning. AADD isn't really defensible.

Later in the book the authors talk about the difficulties of bringing new testers up to speed on a project, and they make this recommendation:

This conversation sparked the idea of a FAQ page, an outstanding issues list, or something along that line that would provide new testers a place to find all of the issues that had been identified but for which the decision had been made not to address them.

Ideas like this should be questioned as to whether or not they scale and if they add unnecessary complexity to the process with too many tools. I can't imagine such a thing scaling well at all, and anytime you create documentation that you'll never use personally, it will not be adequately maintained. I find that keeping these kinds of lists up-to-date is nearly impossible when they're not part of some other process that the maintainer does find useful.

The book did contain a few pleasantly surprising nuggets, though. Near the end when the authors were discussing defect tracking software, they claimed:

If your team is working closely with the programmers and is practicing pair testing as soon as a story is completed, we strongly recommend that you don’t log those bugs as long as the programmer addresses them right away.

It's an interesting difference from bug tracking software proponents, and such pragmatic practices can eliminate a lot of tedious bookkeeping.

All in all, I can't recommend this book either. I pretty much heard it all before and it has been presented much better in the other books I've already mentioned. Maybe if you're a tester that's completely new to the Agile process, you'd find Agile Testing much more useful, but I'd hope you could find a more concise resource. As for programmers, The Pragmatic Programmer and Robert Martin's books are more engaging and are all you're going to need.

More Than Enough Testing

I set out to fill what I thought was a gap in my understanding, and found out that other books I read had already covered what I needed to know. I probably didn't need to read either of these books. It was too much reading about testing and not enough practicing. You only need to read a little bit about testing to understand the benefits and how to go about it, and then it's best to spend the time experimenting with it. Clean Code and Agile Principles, Patterns, and Practices in C# are more than enough on the subject of testing for developers.

The Ivory Tower of Software Development and Messy Reality

What is the best way to learn something that is complex enough and broad enough that you will spend a lifetime mastering it? Some people believe the way to learn something is to first study the fundamentals and proper techniques, and only then move on to doing or creating things. Other people take a different approach and introduce new students to something simple, fun, and engaging right away to capture their interest and imagination before getting into the more technical aspects of the subject.

It's a question of a purist approach or a pragmatic approach—the ivory tower or the open market. This dichotomy has a strong presence in software development, but it exists almost everywhere.

Learning an Instrument

One example of this purist-pragmatist divide occurs when learning to play an instrument. Let's say you wanted to play the piano. You wouldn't have to go to lessons with too many different teachers before you would recognize different beliefs about teaching methods.

The purist approach would involve learning all of the proper techniques first. New student would learn scales, chords, finger position, and a bunch of other fundamentals, and then they would drill and drill and drill until they could do these things perfectly without thinking. They would do all kinds of exercises completely outside the context of real music, and then when they were ready, they could begin playing classical pieces. This is the ivory tower of perfection. Mistakes are eliminated quickly, bad habits are never allowed to take root, and the student is always on the straight and narrow path to musical virtuosity.

The pragmatic approach is a bit different. Instead of starting with technique, the students will start playing songs right away. The songs start out simple, to be sure, and proper technique is taught within the context of the songs that use those techniques. New skills are learned as needed to play new songs, and the focus is on playing real music. The belief is that the required skills will be learned in time; it doesn't have to all be done up front. Certain techniques may not be learned until much later and students may spend more time in areas that they enjoy. The students that want to become much better musicians and challenge themselves more will naturally put in the extra effort to master the more difficult techniques because the music is driving them to do it.

Which method would you prefer? They both have their merits, but I think more people are going to stick with the pragmatic approach because it's more fun and engaging. The purist approach quickly gets boring, and only the most dedicated (or most externally forced) students are going to get to the point where they can play actual music. Most people would give up on it before they got that far.

Programming Purity

Any sufficiently complex field will have this debate between the purists and the pragmatists about what is the best way to teach beginners coming into the field. The analog to the musical purist approach in the software domain is to have students learn all of the proper computer science theory first, before allowing them to write any real software. To a purist, the right computer science education would start with data structures and algorithms, preferably in Haskell or LISP (or both). It would quickly get into compiler theory and lambda calculus so that students would have a firm understanding of the most important parts of computer science.

I have no problem with languages like Haskell or LISP. I think they're great languages, and you'll learn a lot of stuff about programming by learning these languages. But as introductory languages for beginner programmers, they're terribly difficult and a huge barrier to entry.

The purist may claim that difficulty is a desirable trait in a language, and that one of the purposes of introductory programming courses should be to weed out students that don't understand these all important language features and computer science concepts. A computer science education should certify that a professional programmer knows how to write a compiler and an operating system, is proficient in the use of strongly typed languages, and has a firm understanding of closures and first class functions.

Taking this line of thought to its logical conclusion is absurd. Do we really want to require aspiring programmers to learn all the theory first, to certify professional programmers against some arbitrary set of standards, and to only allow professional programmers to write software? What would we test for? How would we differentiate between programmers that are allowed to write compilers from those that are allowed to write payroll software for an HR department? How would we establish which level of the ivory tower each programmer could work in?

Not every programmer needs to know the same stuff to be effective at what they do, and the vast majority of programmers out there don't need to know pure computer science theory to do their work. Besides, thinking that it's even possible to structure a perfect learning path for programming and force every aspiring programmer to follow it is just silly. People don't learn how to program that way.

How People Really Learn to Program

How did you learn how to program? I bet you didn't start by studying memory safety and resource allocation. I've never heard anyone describe their early programming experiences that way. I started out programming in LOGO, making the turtle draw simple shapes on the computer screen. Then I started writing clones of simple games in QBasic. It was fun and exciting, and that is what hooked me in. I wanted to learn more because these early programming experiences were fascinating. I hear a lot of stories similar to mine, but never have I heard a programmer say they got hooked on programming the first time they learned about closures or type safety.

People become programmers for a wide variety of reasons. Not every programmer loves programming, but the best programmers almost certainly do. The desire to program is necessary for learning the more advanced topics of programming. It's too hard otherwise, and the programmers that don't love it are going to find ways around having to think about the hard topics. In the end, that's okay. They'll still have plenty of work to do without the hard stuff.

For those that do love programming, they will be driven to keep getting better, and as a result they will learn the proper techniques and best methods of programming. The fundamentals are hugely important, and one of the best ways to get better is to practice and develop a deep understanding of the fundamentals. Like the piano student that learns the necessary techniques while playing beautiful music, the best programmers will learn the fundamentals while making great software. If they need to learn a certain programming concept to make their software better or to become more effective as a programmer, they'll do it, but the desire to program originated in those early experiences of writing simple programs to do cool things with a computer.

Creating cool programs in an accessible way should be the focus of any introduction to programming. It may be messy and impure, offending the sensibilities of the proprietors of the ivory tower, but it captures the imagination. And the imagination is a powerful force for learning complex subjects and doing awesome things.

The Benefits of Working on a Personal Software Project

If you love programming, then you probably already have one or more of your own software projects in the works. Personally, I find that working on a software project of significant size is one of the best ways to learn new tricks as a software engineer, but that is only one of many reasons to write software in your free time.

Choices

Before getting into the benefits of personal software projects, it's worth taking a brief look at some of the different kinds of projects you could explore. If you wanted to look at embedded programming, you could get yourself a great embedded system for less than $60 (sometimes much less). The most popular choices are Arduino and Raspberry Pi, but there is also the very powerful BeagleBoard and more specialized embedded boards like the TI LaunchPad series.

If you want to get into mobile development, all you need is a smart phone or an iPod Touch and you can do Android or iOS development. You could write a game, a productivity app made just for you, or satisfy some other nagging itch that's been plaguing you.

If you want to explore web development, you have tons of resources available, both online and off, for any number of web development frameworks: Ruby on Rails, Django, ASP.NET MVC, AngularJS, MeteorJS, or something newer and less well-known. There is no shortage of options, and you're bound to come across something new and exciting.

You could even do classic application development in Visual Studio with C# or XCode with Objective-C. There's also system-level programming in Linux. You could study up on your algorithms, look into machine learning, experiment with a new library, or explore what's new in big data. You can even dabble in parallel computing by programming your graphics card with CUDA.

The possibilities are nearly limitless and readily accessible at close to no cost. All of these types of projects are immensely fascinating, but to make any significant progress, you'll have to focus on only one and resist the urge to jump from one to the next like a raccoon chasing shiny objects. If you manage this tremendous feat and settle down to work on a project for an extended period of time, here are some of the benefits that you may experience.

Learn Something New

Starting with the obvious, if you pick a project in an area that you have little experience in, you're going to learn a ton of stuff. By working on a project that's not trivial, you will learn much more than if you had only read about it. Don't get me wrong, I love reading, and I read everything I can. But you won't learn a new language or pick up new skills simply by reading about it. Reading is hugely important and a necessary part of learning, but without something more to cement that knowledge in your brain, you'll quickly forget whatever you read. A project is a great learning tool because you'll struggle with things that seemed simple in the literature, and that struggle and the solutions that you discover along the way will stay with you much longer than what you read.

Practice, Practice, Practice

Projects don't have to only be about new things. You can do a project in a language and framework that you already know well and get some great practice out of it. You could get practice in other ways, like doing programming problems on various websites, but these problems focus on narrow pieces of a very large puzzle. While solving small, contrived problems can be fun and instructive—I thoroughly enjoy it myself—working on a larger software project will give you more practice in the context of something real. You'll also be developing other skills that small problems don't address, like designing the software architecture, deciding on the features you want to include or not, and defining the problems that need to be solved in the first place. Software development is so much more than merely writing code, and doing a real project will give you good practice at all of those non-coding skills.

Take New Tools for a Spin

For most people, using the hot, new programming language or framework at work is strictly forbidden. Unless you're working on a greenfield project in a company that's willing to take a risk on something that's probably not production ready, you'll be working in one of the well-established, mature programming environments at your day job. A personal software project will give you the opportunity to try out an intriguing new language that you've been meaning to learn, but has been off limits for your work projects. Who knows, if you get good enough at it, you could spin up some little utility projects at work to solve immediate problems that you're having and show off the new tool's advantages. One thing could lead to another, and you could end up adding real value to your company's business while getting a chance to play with new tools more often.

A Vehicle for Experimentation

When you've learned a new programming technique or a new language feature, it's probably best to not turn around and try your new-found knowledge out in a production code base right away. You're more likely to tick off your coworkers with all of the mistakes you'll make implementing a design pattern for the first time than actually adding value to a company project. Instead, try experimenting with new techniques on a personal project where you can turn your code upside down and inside out until you've gained a true understanding of the issues involved. Personal projects are also great sandboxes for trying out crazy ideas, just to see if they work.

Portfolio Gold Plating

A substantial personal project is a great addition to a work portfolio. If you've made some cool projects while tinkering at home, it's going to give you much more interesting stuff to talk about in an interview, and it's likely to help your CV stand out in a crowded field because it shows that you have a passion for programming. What's more interesting, saying that you implemented some of the business logic for yet another enterprise payroll system, or that you implemented an embedded control system for a robot that can navigate a maze? If the only kind of work experience you've been getting is the former, a personal project is your best opportunity to show off what awesome stuff you can do. Of course, doing a project for the sake of your portfolio alone won't work because the motivation won't be there, but if you're already doing it because you can't stop yourself, putting it in your portfolio is a bonus.

Glory and Honor

If you manage to build something in your spare time that's truly valuable and noteworthy, and you release it as OSS, you could make a real difference in other programmers' lives. A lot of the frameworks and libraries we use on a daily basis were made in programmers' spare time, and the authors are pretty well-known in the software development community. Fame can be a blessing and a curse, though, and these OSS projects can take on a life of their own. Persevering with a big open source project is a huge commitment, and it will take a lot of energy to see it through. Even more so than portfolio gold plating, you have to have the motivation to succeed with OSS or you won't get very far.

For Fun

This is the reason every passionate programmer does their own software projects, because it is great fun. Exploring new corners of the programming world, experimenting with esoteric and novel programming languages, creating useful things with your imagination and a computer—these are the reasons why programmers get excited about programming and play around with pet projects until the wee hours of the morning. Of all the benefits listed here, none is as important as having fun.

…And Profit

A personal project could eventually grow into a personal business or something more. Plenty of thriving businesses started out as a simple idea and some after hours effort. The amount of effort required to make a viable business is substantially more than messing around with pet projects, but if you think you have a good product that people would be willing to pay for, don't be afraid to charge for it. The worst that could happen is no one buys it. Then you've still got a great project for your portfolio, you learned a ton of stuff, and you hopefully had fun doing it.

It Needs to be Done

Sometimes there's a hole where a software product should be, and if you want it filled, you are the one who's going to have to do it. There's something about your life that keeps bugging you, and you simply can't find the right tool to fix it. You've searched endlessly on the app stores and Google and you've come up empty. Sometimes you don't even need to go to those lengths. Sometimes your problem is so narrow and personal that you know it's only going to get done right if you do it yourself. A custom piece of software can be one of the most satisfying projects because you're solving your own problem with your own skills and ideas. Nothing feels as great as that.

For the Children

If you have kids, working on a project at home can be a great learning experience for everyone. You get to teach them something that you love, and they get to spend time with one of their favorite people while doing something fascinating. Depending on their age, you'll have to tailor the project to something they can handle, but all kids love to build things, create things, and control things. Software projects cater to all of those desires, and it's great for them to learn how to solve problems through their own creativity and imagination. They'll learn how to overcome problems by your example. Watching you struggle with the problems that crop up in programming, realizing that they are solvable, and working through them together is invaluable. Watching their reactions when something finally starts working is incredibly fun and rewarding. The sheer force of excitement from seeing their creations come to life will keep both you and and your kids coming back for more.

Change the World

Big things start from small beginnings. If your personal project is filling a need that extends beyond yourself, it's quite possible that you could change the world. Change doesn't necessarily mean revolution, and the world doesn't necessarily mean all of planet Earth, at least in the beginning. Something that started out as a simple way to connect and share with other people or to optimize your health and lifestyle could grow into The Next Big Thing. Who knows? You'll never find out unless you start that project. Go learn something, have some fun, and change the world.

There Must Be A Better Way

As developers, we often try to defend our over-engineered and complicated solutions, and indeed, every problem has some amount of irreducible complexity. But there is often a simpler solution if we take the time to look for it. Part of good software development is believing in and spending the time to look for that simpler way. Don't resort to overly complex implementations just because the problem seems to call for it. The simple solution may not seem so shiny, but there is elegance in simplicity. It may not feel like you're practicing good engineering when you come up with a solution that looks too easy, but often times the simple solution is more robust and resistant to buggy edge cases.

In engineering and the sciences, simple is normally equated with elegant. A simple solution is easier to understand, maintain, and extend than a complicated one. The difficulty comes in finding a simple solution. It's hard work to understand a problem so thoroughly that the solution is made to look easy and obvious, but that is the mark of a good solution. All of that hard work may not be apparent in the end result, but it provides much more value than a complicated mess of code that people look at in awe because they can't believe that it even works.

Sometimes it seems like the problem is too complicated to warrant a simple solution. Other times the immense architecture of the complicated solution is a siren call that lures you in and blinds you to simpler alternatives. Finally, there are times when you stare in disbelief at the simplicity of the solution you came up with and convince yourself that such elegance isn't possible. Instead, you discard the easy option and pursue a more complicated path. I experience at least one of these traps with every engineering problem I work on, and it can be difficult to avoid them. Here are some things that I've found helpful in the quest for a better way.

Resist Over-engineering

Some programmers seem to revel in the intricate architectures that they create, claiming that the problem is so complex that a simpler solution is not possible. But what they've really created is an unmaintainable mess—a monstrosity that will collapse under its own weight before it ever gets close to shipping. You look at this pile of code and think, "there must be a better way," but the original architect can't see it because he's locked into the path he took and believes that the extra complexity adds necessary flexibility or extensibility.

Finding the simpler solution requires a completely different perspective and a willingness to solve only the problem at hand. A solution should be only as complex as the problem warrants. If the problem actually needs the added flexibility of the more complex solution, then it may be appropriate, but flexibility shouldn't be added for its own sake. Joel Spolsky railed against the unnecessary complexity touted by architecture astronauts when he praised pragmatic programmers that reject complicated frameworks and instead get stuff done. His advice is more pertinent than ever.

The basic idea is that design patterns, threading, templates, and other advanced programming techniques are complicated tools that should only be used when it is absolutely necessary. Don't use them just because you can. Use them when not using them will end up being more complicated. Deciding when to use the Factory pattern is a prime example of this concept. In theory, you could use the Factory pattern everywhere you instantiate an object, but that would clearly be overkill. On the other hand, situations arise where this pattern is genuinely useful. If you are instantiating similar objects from a class hierarchy in many places in your code, the Factory pattern will likely simplify your code instead of complicating it. That's when you want to use it.

I've been playing around on Exercism.io lately, and I've seen plenty of interesting examples of over-engineering in the solutions that programmers have submitted there. I thought about showing a couple examples here, but I don't want to single anyone out. Suffice it to say, some programmers submit verbose solutions with multi-level class hierarchies or delegation for simple problems like RNA transcription or converting decimals to Roman numerals. (Examples like these are plentiful on any coding site. Codewars.com and Project Euler are two more programming problem sites that allow you to compare your solutions with other programmers, and over-engineering is easy to find.)

If these programmers write such complicated code for simple problems, what do they do when the complexity of the problem scales up? I can only imagine. Complicated architectures have mental costs, and we only have so much brainpower to work with when programming. Why waste any of it on the overhead of an over-engineered, over-architected solution that solves more than the problem requires? It doesn't add clarity, and every time you have to deal with that code, it drains your mental reserves.

Find the Simple in the Complicated

Even after resisting the urge to over-engineer, the simple solution may still be elusive. Once the code is working, it will still be possible to make it better. I have not written or read a first-cut of a piece of code yet that couldn't be improved. That doesn't mean that every piece of code needs to be optimized, but if a part of the program needs to be extended or has issues that need to be resolved, finding a simpler solution is almost certainly an option.

I'm constantly amazed by the simplifications I miss when programming. While I didn't feel comfortable showcasing other programmers' code, I can certainly use my own, so here's my solution to the RNA Transcription problem on Exercism.

class Complement
  TO_RNA = {C:'G',G:'C',T:'A',A:'U'}
  TO_DNA = {C:'G',G:'C',U:'A',A:'T'}

  def self.of_dna string
    transcribe string, TO_RNA
  end

  def self.of_rna string
    transcribe string, TO_DNA
  end

  def self.transcribe string, transcription
    string.each_char.map { |c| transcription[c.to_sym] }.join
  end
end

The problem is fairly simple. It asks you to create a couple methods that return the RNA sequence for a DNA sequence (Compliment.of_dna) and vic versa (Compliment.of_rna). I thought this solution was pretty straightforward. I have named constant hashes for converting to RNA and to DNA, I use a generic transcribe method for both of the conversion methods, and the two conversion methods even read nicely, almost like English.

I was happy with the solution, but upon reviewing other submissions, I found that Ruby actually has exactly the method that I created as self.transcribe already included as part of the String class. I could simplify my solution quite a bit by using the String#tr method:

class Complement
  RNA = 'CGAU'
  DNA = 'GCTA'

  def self.of_dna string
    string.tr DNA, RNA
  end

  def self.of_rna string
    string.tr RNA, DNA
  end
end

Using the built-in method shaved four lines off the code and simplified the RNA and DNA constants. In the process of reviewing other people's code and striving to simplify my own, I learned about a Ruby String method that I hadn't known before and had some good practice expressing the essence of a particular problem. It's a great skill to cultivate.

Nearly all of the other problems I've solved have followed this same pattern. I try to come up with as simple of a solution as I can that directly expresses the core of the problem, and then I learn an even better way of doing it by reading other people's code. The same principle holds true for any programming that I do. If I need to find a better way of doing something, I can be sure that it exists; I just have to find it. The combination of Google and reading good programming books helps in the search for the problems I deal with outside of coding practice sites.

Don't Worry That the Simple Isn't Complicated

It may not feel like doing real work when you come up with a simple solution. You may feel guilty for producing something so obvious after a large investment of time! That's normal. Simple solutions are not something to be avoided because our egos believe that the problem warrants something more complicated. Don't confuse simple with simplistic. Simple solutions are more elegant, and are often more robust to edge cases and changing requirements than the overly-complicated alternatives. Simplistic solutions don't address all of the requirements and fall short of the problem's inherent complexity. We want simple solutions to complex problems.

If the simple solution works, don't worry that it looks small compared to the architectural edifice of a multiple-inheritance hierarchy. Coming back to Joel, he doesn't mince words on this issue:

You see, everybody else is too afraid of looking stupid because they just can’t keep enough facts in their head at once to make multiple inheritance, or templates, or COM, or multithreading, or any of that stuff work. So they sheepishly go along with whatever faddish programming craziness has come down from the architecture astronauts who speak at conferences and write books and articles and are so much smarter than us that they don’t realize that the stuff that they’re promoting is too hard for us.

Don't go along with the architecture astronauts if a simple solution will suffice. They may appear smarter, but they're also so deep in the complexity that they've created that they can't see that it will risk wasting everyone's time. A simple solution that works now and ships now is infinitely better than a complicated solution that will handle every possible future use case or additional feature but never ships.

Besides, complicated solutions always make me suspicious. A simple solution is a sign that the the problem is well-understood and the developers have thought hard about taking the right approach without generating any waste. Great programmers are like great teachers who really understand their subject and can teach it with a clarity of thought that makes complex material look easy. Great programmers will not complicate a design for the sake of over-engineering. They will search for the right solution that is appropriately simple and directly solves the problem. They don't worry about impressing others with complicated monstrosities when working software will prove their worth.

To be a great programmer, start believing that there must be a better way. Then find it.

Momentum

"An object at rest tends to stay at rest, and an object in motion tends to stay in motion." Isaac Newton's First Law of Motion seems to apply as well to projects as it does to physical objects. In my experience it is much easier to continue working on a project and keep it moving if it already has decent momentum. It is much harder to make any progress on a project that is languishing from lack of attention, or has hit a barrier that has stopped it completely. To keep a project that you care about moving, you need to at least maintain momentum, and preferably build it up.

In classical mechanics, the measurement of momentum of an object is a simple calculation consisting of the mass of said object multiplied by its velocity:

p=mv

The equation couldn't be simpler, but it has powerful implications in physics. Those implications can also apply to projects.

Specifically, a project has properties that could be considered its mass and velocity. The mass of a software project could be the amount of code, design assets, and documentation that have been created for it. If the project has been released and is out in front of real users, the mass could also be the number of people using it. If the users are actually customers paying for the software, the mass could be the amount of revenue the software is bringing in. In any case, the mass of a project is most likely growing over time. Since the mass is increasing, according to the equation, momentum should also be increasing, but there is that other term to consider.

The velocity term is a vector that has a direction and a magnitude. The magnitude of the velocity (a.k.a. the speed) of a project is simply how fast it is moving over time. How quickly are features being added, bugs getting fixed, or interfaces getting polished? How fast are infrastructure and various support mechanisms being added to the project so that it can survive in the wild? Those are some of the things that determine the speed of a project.

As for the direction, that is determined by how the project is progressing towards the goals set out for it. Two things need to be considered here. The project could either be headed in the direction of the project goals or not, and the project goals could either be in the right place or not. Ideally we want the project goals to be in the right place and the project heading towards them, but that's not always reality.

We've now come to the first insight we can gain from this mental model. (It's obvious, but that's the way insights are sometimes.) If you realize a project is heading in the wrong direction or the goals are way off-base, then how easily the project can be course-corrected will depend on its mass and its velocity. The larger a project is or the faster it's moving, the more energy it will take to change its course because it has more momentum. Likewise, the larger the course correction needs to be, the more energy it will take to get it back on track. Small adjustments can be made even to large or fast-moving projects relatively easily, but if you have to turn a large project completely around, it is going to take a lot of effort.

Something else that happens to projects over time, other than needing course corrections, is that they grow. Much of the energy that a team puts into a project gets converted to mass in the form of the aforementioned materials, customers, and revenue. All of this accumulated mass makes projects more sluggish as they grow. It takes more and more energy to keep a growing project moving fast, which is why it seems physically impossible to make progress on huge, crusty projects with any kind of speed. In contrast, a greenfield project has almost zero mass, so it feels like making lightning fast progress takes no effort at all.

Those greenfield projects don't stay small for long, though. A new project that's making good progress will quickly increase in mass. As it grows, another force will come into play that is alluded to in Newton's First Law of Motion—friction. Since a project is never built in a vacuum, it will have to overcome all kinds of friction to stay in motion. As a project grows, its surface area will also grow, increasing the frictional forces on it.

Even projects that you wouldn't think would encounter more friction as they grow will, in fact, do so. For example, this blog seems to be adding friction as I write more and more posts. I didn't anticipate that. As I've learned things about writing and added new features to my later posts, I am acutely aware of the benefits of revisiting some of my older posts and updating them. I should also refresh my memory of things that I've already written. Both of these frictions continue to grow as I write more, and I'm sure other frictions will arise as I keep going. I also continue to deal with the multitude of distractions from everyday life that are pulling me in directions other than writing.

One way that I've found to keep the momentum up with writing—something I committed to from the start—is to write every single week, no excuses. I normally do even better than that by doing something blog related every single day. It's not always writing; I do that two to four nights a week. The rest of the days I'll make notes in Trello when an idea strikes me or I'm reading something and come across a good example for a future post. I also take time to think about my writing. It may not all seem like momentum, but it all adds up to a blog that keeps moving. It has worked pretty well for me.

I've found that that mentality works well for any kind of project, personal or work-related. Personal projects have all kinds of frictions, not the least of which is probably work and the energy drain that comes with it. It's so much easier to come home and veg out in front of the TV or drift on the internet than sit down and work on a project, but none of that will keep the momentum going, even if you care deeply about the project. The same is true for work projects. Frictions abound with meetings, email, and other (sometimes necessary) distractions.

The best way to overcome these frictions, whether for work or personal projects, is to do something for the project every day. It doesn't have to be a big thing every day. Some days are so packed with meetings or you're so exhausted when you get home that the best you can do is fix one simple little bug or think about the next feature for ten minutes, fifteen minutes tops. Do that. Even a small bit of progress will keep the momentum up on a project so that it's still moving when you have the time and energy to do something big for it.

The last thing you want is for the project to stop completely. Then you have to overcome the most devastating of all forces—static friction. Keeping momentum going is so much easier than starting up a project from a dead stop, especially as the project gets bigger (remember the project's mass). If you want a project to keep going, do at least one thing for it every day because the momentum will help carry your project along. Momentum is a powerful thing.

Leveling Up

As a programmer, I'm always trying to improve my abilities, to level up my programming skills. Reading a lot helps, but that only provides part of what's needed to become a better programmer. Reading about programming gives you the knowledge of what is out there and available to use, but it doesn't give you the ability to actually use any of that knowledge when you need it. To have the knowledge ready at your fingertips requires cold, hard practice.

I've written recently about the importance of knowing the basics, and the main reason for it was that if you know the basics, you can derive the higher level abstractions that are so useful in programming without having to memorize them all, giving you access to substantially more tools than you otherwise would have. But there is another reason that knowing the basics is so important. If you know the basics of programming so well that you don't so much recall them when needed as they are emitted from your fingertips into your text editor before you even thought to use them, then you will be able to think at a higher level while programming than would otherwise be possible.

Programming, Fast and Slow

Lately, I've been listening to the StackOverflow podcasts in the car. I started from the beginning and have been going through them back-to-back. It's a Joel and Jeff overload, but they have some fascinating discussions pretty regularly. One discussion in Stack Overflow podcast #57 was on this very topic of good programmers being able to belt out basic programs quickly. It starts off with a question from a StackOverflow user who is lamenting the fact that StackOverflow seems to reward programmers that can answer questions really fast, basically promoting the fastest gun in the West without much regard for quality.

Regardless of whether or not that's true about incentives and quality, Joel took the opportunity to start analyzing what makes a good programmer and how to figure out if any given programmer is potentially good. He describes how he used to use an interview question with a very basic problem that could be coded up in a few if statements. He needed a way to filter out people from the dot-com crash that were looking for jobs but didn't actually know how to program, and he was hoping this simple problem would be a good first-stage filter. Here's what he did and what he found, edited a little for clarity:

It's sort of something like ... is John older than Mary?  <Laughter>  It's a little harder than that!  …[W]hat happens when I ask this question is that everybody gets it, but some people take a long time to think it out, and they're carefully thinking it through, and they really are struggling, and some people just write really fast and they're done.  And then we can move on to more advanced things.  And, it suddenly occurred to me, that there was a very high correlation between the people that I hired, and the people that could solve this problem as fast as they could write.

I don't think this means that you can study and practice the fundamentals exclusively, and magically you'll be able to solve the hardest programming problems. It doesn't work like that. I can certainly believe that great programmers are more likely to have a solid grasp of the fundamentals, so practicing the basics is a prerequisite that opens the door to higher levels of proficiency.

Joel continued with a fascinating story about a math professor teaching calculus:

Serge Lang is a math professor at Yale who is very interested in math education. Serge Lang, at the beginning of freshman calculus at Yale, gave people, for no credit, … told all the students to take out a piece of paper, and he put a fairly simple equation up on the board and said, "reduce this to its simplest terms." So this is basically 9th grade algebra. Then after like 30 seconds he said, "stop!" and took all their pieces of paper.

Some students were able to reduce this algebra equation to its simplest form as fast as they could write, and some of them really had to think about it and really had to work on it. … And he said that the people that could do it quickly was an almost perfect predictor of who would get an A in the course. In other words, that was as good a predictor of math ability as an entire semester in calculus with problem sets every week and probably two mid-terms and a final. 

This kind of predictor is fascinating, but after thinking about it for a while, it makes total sense. The students who did well in the course were nearly always the same students that did well in the simple algebra problem. That correlation doesn't necessarily mean that some students performed better because of some inherent proficiency in math. It could also mean that they did a much better job of internalizing the fundamentals of algebra so that by the time they needed to use algebra for calculus, they knew it backwards and forwards and could spend their mental energy on the new ideas of calculus.

Meanwhile, those students that struggled with the simple algebra problem probably continued to struggle with the algebra that they needed to use throughout the course. Because they never made the basic operations instinctive, they couldn't grasp the higher levels of thought necessary to learn calculus. Derivatives and integrals were beyond their reach because they still needed to adequately learn how to simplify and balance equations.

Pretty much all of mathematics depends on knowing lower level operations and theory well to reach higher level theory. This process is obvious at the lowest levels of counting, arithmetic, and number systems, but it is also true at the highest levels of mathematical theory. Mathematicians often have a sense that a theorem is true long before they can prove it, and much of the work of proving it is building a foundation of simpler proofs and machinery to use to support the final theorem and discover a valid proof for it. Without the hard work of learning the basics, the higher-level proof would be unattainable.

From Dancing to Chess

This same concept of mastering the basics applies in any learning scenario that I can think of. Take dancing as an example. Most styles of dance have a basic step or basic set of moves that define that style. Swing has a basic step. The Tango has a basic step. Ballet has a basic set of movements. All of the other moves in a dance style branch off from that basic step.

I took some ballroom dance classes in college with my then-girlfriend, now-wife, and for each dance style we learned, we were constantly encouraged to practice the basic step until it was second nature. We should practice it until we could carry on a deep conversation without ever missing a step. It had to be automatic. If we had to think about what our feet were doing for the basic step, we had no chance of learning the fun dips and spins of the more advanced steps. Once we knew the basic step, it acted like a safety net that we could always fall back on if we got lost in a complicated move or made a misstep, and it gave us a lot of confidence to learn more challenging stuff.

Okay, enough about dancing; how about playing an instrument? The same idea applies. If you learn the basic chords on the guitar or the bow strokes and fingerings on the violin so well that you no longer have to think, "Okay now this finger goes here and this one goes here and I have to move my arm like so…," then you can focus on the music you're playing and actually think ahead to the measures you'll be playing next. When you've built up the muscle memory and your fingers are doing all of the thinking for you, that's when you can level up and really play.

Playing chess is another example where reaching higher levels requires knowing the fundamentals by heart. The best chess players in the world don't look at a position and see a bunch of pieces that need to be analyzed individually. They see patterns of moves, pawn structure, and collections of interacting pieces that allows them to see many moves ahead. All of that knowledge and ability to see patterns comes from thousands of hours of practicing the fundamentals of chess. They know the tactics of pins, forks, and skewers for all of the pieces. They thoroughly understand the implications of doubled, isolated, and passed pawns. They immediately see strategic advantages of space, outposts, and initiative.

Chess masters can clobber amateurs so quickly and decisively because they are so well-versed in the fundamentals. They don't have to think about their immediate next move because they can see the combinations in a position and know the consequences of different moves by instinct. All of their hard work has paid off in an ability to operate at a much higher level of chess thought.

Take Time Out to Level Up

All of these examples have at least one thing in common—lots of practice. It's like level grinding in a role-playing game. If you reach a point in the game where you can't seem to make any forward progress, you probably need to go off and fight a bunch of enemies, gain some experience, and grind out a few levels. When you come back to the point where you were stuck before, it will be much easier after you've leveled up.

Practicing the basics until they are second nature, until you no longer have to think about them at all to use them, is key to leveling up as a programmer. This is not as easy as it sounds. It can be incredibly boring if you don't use a wide variety of practice tools and incredibly frustrating when it feels like you're not making progress or are even regressing. That's why plenty of programmers plateau a few years out of school. Practicing is hard, but necessary, and the programmers that practice regularly and effectively will reach a much higher level of programming competence. They will be the ones that can solve the hard problems in programming now and in the future. Are you leveling up?

Test in Moderation

Software testing is a huge topic, and intense debates are going on all the time about what is the best way to do software testing. Test Driven-Development (TDD) has built up a strong following over the past 15 years, and with good reason. TDD can have a lot of benefits if done well. It also has its downsides, and if taken to an extreme, TDD can cause more harm than good. Programmers are justified in pushing back on the more extreme forms of TDD, but we don't want to go to the other extreme, either. Testing is a necessary part of a high-quality development process that increases the odds of releasing highly-functional software. We need to do testing in some form. The real questions are, how should we test and how much?

The intention here is not to focus solely on TDD, although that will be difficult because it's something most programmers are familiar with. What I really want to talk about is testing in general, and TDD is but one piece of the complete testing picture. The rest of the picture includes things like usability testing, code reviews, and quality assurance (QA) testing. Proponents of TDD, when at their most extreme, claim that TDD subsumes and encompasses all other types of testing. They take every possible type of testing, define and categorize them, and put them all under the banner of TDD. It may be a convenient title, but in practice most programmers think of TDD as focusing on unit and integration testing because those are the tests that are most easily automated.

When referring to TDD, the concept I'm thinking of is essentially the same as the one used by Steve Freeman and Nat Pryce in Growing Object-Oriented Software, Guided by Tests:

When we’re implementing a feature, we start by writing an acceptance test, which exercises the functionality we want to build. While it’s failing, an acceptance test demonstrates that the system does not yet implement that feature; when it passes, we’re done. When working on a feature, we use its acceptance test to guide us as to whether we actually need the code we’re about to write—we only write code that’s directly relevant. Underneath the acceptance test, we follow the unit level test/implement/refactor cycle to develop the feature[.]

I'm using 'integration' whereas they use 'acceptance,' but they're pretty interchangeable. It's clear from this explanation of the TDD process that acceptance tests do not include getting the user interface in front of actual users, getting the code in front of coworkers, and getting the product in front of physical, human testers. That's okay. The definition of TDD should be focused, but we shouldn't forget about the rest of the testing picture.

In the Grand Scheme of Things

One great way to step back and see the bigger picture is to do a postmortem on a project (referred to as a retrospective in agile development). Despite the name, don't wait until the project is finished (or dead) to do a postmortem. You can learn a lot from looking back on a project after certain milestones are reached, like major releases, and many software projects are never truly finished anyway.

When I was working for an ASIC design company, we sometimes did a postmortem right before we would release the second revision of silicon for prototyping. This was a perfect time to take a look back at what happened during the previous cycle of silicon evaluation because we were about to send off the design for fabrication again. It would be 6-8 weeks before we'd see the results, and it would be very unlikely that we could fix any mistakes we found after pulling the trigger.

I remember one postmortem in particular that was for a project with a high profile customer. We were aiming to complete the design in two and a half spins—that means the third revision would only change the metal interconnect layers of the design—and as always, we were schedule constrained to get it right. Before releasing the design for the second spin we took a look back at all of the bugs that we discovered during the first silicon evaluation. We had found 35 bugs (pretty respectable for first silicon), and they had varying degrees of severity and scope.

While we didn't discover anything that caused us to delay the next release, we did learn some interesting things about the source of the bugs. Part of the postmortem exercise involved figuring out how we could prevent those bugs that we found in the future. I can't remember the exact numbers, but we found that a significant number of those bugs, like more than half, could not have been prevented with better automated testing and simulations.

Some bugs we definitely should have caught. They were obvious after we had gotten silicon back, the fixes were obvious, and the simulations required to catch the bugs were obviously missing. It's your standard face-palm kind of stuff.

Other bugs had a root cause that didn't originate in the implementation, but in the specification. These were the inevitable misunderstandings of how the product should work, either on our part, the customer's part, or the communication between us. We continually strove to reduce these kinds of mistakes, but there's always room for improvement when it comes to product specifications and customer communication.

Then there were bugs that we could not have predicted would be an issue until we had physical silicon in our hands to test. Similar to most software development, we were doing something that had never been done before, at least by the collection of engineers participating in this project. We knew going in that there would be issues with the design, but we couldn't know what all of them would be. We did try to mitigate the risk by making the design as flexible as possible in the areas where we thought there would be trouble, and we planned for the inevitable bugs with the two and a half spin schedule. It's actually a tribute to the design team that there were so few bugs in first silicon and most of them were of this unpredictable variety.

We ended up shipping revision B2 as planned, and it was one of the most well-executed projects I've had the pleasure of working on. However, the take-away for this discussion is that in spite of our quite rigorous automated testing, we didn't catch every bug that we needed to, nor did we have any hope of doing so even if we devoted ten times the effort to testing simulations.

Imperfections in Testing

The equivalent automated testing in the software world is integration and unit testing, whether it's TDD or not, and Steve McConnell has some interesting statistics on the effectiveness of different types of testing in Code Complete 2:

.. software testing has limited effectiveness when used alone—the average defect detection rate is only about 30 percent for unit testing, 35 percent for integration testing, and 35 percent for low-volume beta testing. In contrast, the average effectiveness of design and code inspections are 55 and 60 percent.

The defect detection rates for unit and integration testing are amazingly low. Similar rates are probably applicable for ASIC simulation testing, and that is why we would have had to put in so much more effort to catch the obvious bugs that got through. During development those bugs are not obvious, and you have to write a ton of extra tests and cover cases that aren't broken to find the few remaining cases that are hiding the bugs. Other types of testing can make those bugs more obvious.

One argument against TDD is based on this issue of missing obvious bugs. The argument goes something like this. If you write the minimum amount of test code to produce a failing test and then write the minimum amount of production code to make the failing test pass, what you’re actually doing is minimizing the amount of production code you write. In the end, you’ll probably write much less error checking code, and you’ll forget to write error checking code for certain conditions or in certain circumstances, letting bugs get through the development process.

I don't think this argument only applies to TDD. You have to think of the error condition where a bug might exist whether or not you're practising TDD. The TDD approach just changes the way you handle error checking during development. If you think of a possible error condition in your code, you write a test for it first to make the failure a reality, and then write code to fix it. Humans are notoriously bad at predicting failure so it may be better to make sure the bug is real before trying to protect against it. TDD doesn't alleviate developers from needing to be diligent about error checking. Although it may help eliminate unnecessary error checking code.

Another argument against TDD came up in a discussion on the Stackoverflow podcast #38. (I know it's old, but history can still be relevant.) Joel and Jeff talk about unit testing, TDD, and the SOLID principles. Joel questions whether TDD and SOLID are agile or not. It's an interesting thought, and I can see how he would come to that conclusion if these practices were taken to their extreme: 100% unit testing, TDD for every single line of production code, and all the SOLID principles applied to every class in the system. Such a code base would be an incredible, unmanageable mess. Especially the interaction of unit testing every single little thing about every class while separating classes into thousands of tiny single-function elements causes the entire design to be very rigid and fragile to change. I shudder at the thought of it.

While it is useful to have an automated test to check code that you've written so that you don't have to keep checking it manually all the time, that does not always trump other factors. Sometimes it is a piece of code that will never break again, and writing the test is way more complicated than the actual code. In some cases it makes sense to not write the test and instead write the code, check it manually, and be done with it. As a general rule, if a test is hard to write—much harder than writing the actual code—think about whether or not to actually write the test. It would have to provide some benefit, like if the code is especially risky, before it's valuable to write and verify a complicated test.

I like the balance that Michael Hartl strikes in his Ruby on Rails Tutorial. Sometimes he uses TDD, sometimes he writes integration tests or unit tests after coding the functionality, and his assertions have a light touch that leaves enough room for the code to be flexible

Expanding the Testing Repertoire

When writing tests, it's best to KISS. If you get too wrapped up in testing it's easy to go overboard, and that can waste a lot of time. There is some intersection of tests that catch bugs and code that is the source of bugs (or code changes that create bugs). That sounds like a venn diagram to me.

 

If you write tests for 100% code coverage, you're creating a huge number of tests and spending the corresponding huge amount of time writing those tests when some percentage of tests will never ever fail, and are thus useless.

Another way to think about this problem is to consider how many bugs are caught as you ramp up the amount of time you spend using a given testing style. For unit testing, as you write more tests it gets harder and harder to find more bugs, and tests start to overlap in the bugs that they catch. This is why 100% code coverage doesn't mean 100% of bugs caught; it only means about 30% of bugs caught. Over time, testing efficiency tends to look like this:

At some point, putting in more effort doesn't give you any more progress. This trend is true of all types of testing. Whether it's unit testing, usability testing, code reviews, or QA testing, testing efficiency is going to saturate at some level below 100% bug-free because different testing styles will catch different classes of bugs. Instead of taking one testing style to the extreme, it's better to mix different testing styles together so that you're hitting your software from multiple angles. If you have a fixed amount of effort that you can dedicate to testing, different mixes of testing styles will result in varying degrees of quality in the final product.

This chart represents what the space of testing efficiency might look like if you could spread out all variations in the mix of testing styles on one axis. Somewhere in that curve would be a point where you did 30% automated testing, 20% code reviews, 10% usability testing, 20% QA testing, and 20% other miscellaneous testing. Maybe 100% automated testing is near that valley on the left. It's really representing a multi-dimensional space on a single axis, and it's purposefully vague because the maximum testing efficiency is going to heavily depend on everything—the product, the problem domain, the experience and skill of the team, the schedule, the customer, and the list goes on. Experience is probably the best remedy for this dilemma because with experience you'll learn what a good mix of testing is for the contexts that you generally develop in.

One thing's for sure. It's not worth hammering on one testing style to the detriment of all others. That's a terrible waste of time. It's better to back off the 100% code coverage goal—trust me, it's a mirage because you're still far from 100% input coverage—and look for other ways to test more effectively. Automated tests are great, but they should be used judiciously where they give you the most benefit by making sure your stuff isn't fundamentally broken. Then, spending some time getting your product in front of real users or having another developer look over your code and talking them through it will provide much more value for your testing time than writing another test that verifies that, yes, your getters and setters do indeed work. Like all things, testing is best done in moderation.