Tech Book Face Off: Growing Object-Oriented Software, Guided by Tests Vs. Agile Testing

It's been nearly eight months since I've done a Tech Book Face Off. That is way too long. Granted, I had been reading plenty of other stuff—math books, physics books, and econ books—but it's high time that I got back to some software development books. One area where I thought there was still somewhat of a hole in my understanding was automated testing, so I picked out a couple books on testing that I thought would be good: Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce and Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin and Janet Gregory.

VS.

I'm not going to lie. Getting through these books was kind of a slog. I had hoped to learn lots of great new insights on how to go about automating tests and working testing into the software development process without going crazy. I found out that pretty much everything I needed to know was already covered in other books I had read, especially Clean Code and Agile Principles, Patterns, and Practices in C#, two books I've reviewed previously.

While I have some misgivings about panning anyone's work because these authors obviously put tons of time and effort into their books, I don't want to mislead any readers about the value that I got out of these books. After all, these reviews are supposed to give my assessment of whether a pair of books is helpful or not. Sometimes both books are good and complement each other nicely, sometimes one is clearly better than the other, and sometimes I don't find either one very useful. I try to describe those opinions as respectfully and honestly as I can while providing some of the reasoning behind those opinions. With that in mind, let's take a look at these two books on agile testing.

Growing Object-Oriented Software, Guided by Tests

The premise of this book is something I totally agree with. In many ways software grows and evolves over time. Freeman and Pryce nicely described three distinct ways that a code base grows and changes through refactoring:

Breaking out: When we find that the code in an object is becoming complex, that’s often a sign that it’s implementing multiple concerns and that we can break out coherent units of behavior into helper types. … Budding off: When we want to mark a new domain concept in the code, we often introduce a placeholder type that wraps a single field, or maybe has no fields at all. As the code grows, we fill in more detail in the new type by adding fields and methods. With each type that we add, we’re raising the level of abstraction of the code. Bundling up: When we notice that a group of values are always used together, we take that as a suggestion that there’s a missing construct. A first step might be to create a new type with fixed public fields—just giving the group a name highlights the missing concept. Later we can migrate behavior to the new type, which might eventually allow us to hide its fields behind a clean interface, satisfying the “composite simpler than the sum of its parts” rule.

I find that a lot of my software design progresses in these ways. The architecture of a system emerges over many iterations, step by step. This constant flux is a characteristic of Agile development in particular, and it can be an issue for teams and management that are new to it. The authors make some good points about that initial discomfort as well:

[I]ncremental development can be disconcerting for teams and management who aren’t used to it because it front-loads the stress in a project. Projects with late integration start calmly but generally turn difficult towards the end as the team tries to pull the system together for the first time. Late integration is unpredictable because the team has to assemble a great many moving parts with limited time and budget to fix any failures. The result is that experienced stakeholders react badly to the instability at the start of an incremental project because they expect that the end of the project will be much worse.

The goal with Agile development is to not end up in crunch time at the end of the project, but instead spread the chaotic stuff more evenly over the full length of the project, making it more consistent and manageable in the long run.

The first part of the book was much like these excerpts with clear, well thought out discussions of TDD (Test-Driven Development). The second part of the book was an extended example of developing an internet auction bidding program with Agile testing principles, and that is what I had a hard time with. It became increasingly difficult to stay interested as the example went on.




There is something to be said for working through a non-trivial example to better show how a process would work in a more realistic project, but the example that they used seemed too complicated to me. Maybe it was because I haven't written in Java since college and I'm not familiar with any Java frameworks. However, I hadn't done much programming in C# before reading Agile Principles, Patterns, and Practices in C#, and I didn't have any problems with the examples used in that book. In this Java example, the authors seemed to pick a bunch of graphics and networking frameworks that required some prior familiarity to understand what was really going on. For all I know, they are common Java frameworks, but using them served to narrow the audience that could follow along. The complexity of the example and the prior knowledge required obscured the message that the authors were trying to get across.

Once the complexity of the application increased, the big ideas tended to get buried in the details of working through the code. The main point that the authors kept coming back to was that encapsulating related functionality enabled programmers to understand the code quickly and change it easily in one place. This is programming nirvana, but it was hard to see how that encapsulation developed through all of the details of their code samples.

Another issue that came up was that after a couple of tasks were completed, it wasn't clear that it was useful to continue with the example. New concepts and benefits of TDD were no longer being demonstrated; we were just plowing through the rest of the example using the same process we already know about. The danger with a long example is that the focus changes from teaching concepts using a demonstration to completing the example for its own sake. It's a difficult balance that I'm well aware of. You start off with good intentions, but as you get further into the example, it takes on a life of its own. After a while you pull back and wonder how you got where you are, and are you really saying what you mean to say anymore. The overarching ideas have gotten lost in the details of the example. In this case, the goal should have been to show how software can grow efficiently while using TDD, not the gritty details of how to implement an auction sniper.

Even though the extended example was a drag, the authors still had some nice insights into practical testing:

Some TDD practitioners suggest that each test should only contain one expectation or assertion. This is useful as a training rule when learning TDD, to avoid asserting everything the developer can think of, but we don’t find it practical. A better rule is to think of one coherent feature per test, which might be represented by up to a handful of assertions. If a single test seems to be making assertions about different features of a target object, it might be worth splitting up.

I like the pragmatic nature of this recommendation. Dogmatically splitting up tests so that each has a single assertion only serves to extend your test time and bloat your test code. When a set of assertions focus on one feature and have the exact same setup and tear down, they should be combined into one test.

Overall, Growing Object-Oriented Software, Guided by Tests was okay, but I can't recommend it. There are much better books out there, and this one didn't add much to my understanding of TDD. If you're an experienced Java programmer that happens to work with OpenFire, Smack, Swing, and the other libraries that they used in their example, you may get more out of it. Otherwise, I would pass this one by.

Agile Testing: A Practical Guide for Testers and Agile Teams

After getting through the last book, I was hoping for something a little more engaging with Agile Testing. This book was written by a couple of testers who have worked on a number of Agile teams and gave their perspective on how testers could fit into the Agile process after the programmers on the team have adopted TDD. Crispin and Gregory make the case that there are many other types of testing that still have a place in improving software quality, and I strongly agree. Automation with unit and functional testing only scratches the surface of the testing that needs to be done for a product, and having dedicated testers on the team that offer their unique perspective and talents is invaluable.

I did find some insights that resonated well:

If you’ve worked in the software industry long, you’ve probably had the opportunity to feel like Lisa’s friend. Working harder and longer doesn’t help when your task is impossible to achieve. Agile development acknowledges the reality that we only have so many good productive hours in a day or week, and that we can’t plan away the inevitability of change.

This is, of course, one of the key arguments for adopting an Agile process, and you'll see similar statements in any Agile book you read. Most of the book was much like this quote—reasonable and well thought-out—when taken in isolation, but taken as a whole, it was excessively verbose and redundant. The authors were constantly discussing the benefits of automated testing, the use of this or that tool during the testing process, or how you really don't want to make an iteration a mini-waterfall process. I felt like they could have easily covered the same amount of material in less than half of the 576 pages they took to do it.

The book was organized into five parts:

• Introduction to Agile Testing
• Organizational Challenges in moving to Agile Testing
• The Agile Testing Quadrants and the different types of testing that makes up each quadrant
• Automation of any testing that warrants it
• An Iteration in the Life of a Tester and how testers fit into an Agile development environment

At a high level it seems well organized, but there was so much overlap between these sections that much of every chapter was repetition and cross referencing. After a while the periodic tidbits of wisdom didn't seem worth the trouble of wading through pages and pages of generalities. Even the beginning of each chapter had an extra repetition of the rest of the chapter with a mind map that exactly duplicated the chapter's section headings. Maybe that's useful to some, but it just added pages to the book for me.

Some of their points were surprising as well. For example:

“AADD,” Agile Attention Deficit Disorder. Anything not learned quickly might be deemed useless. Agile team members look for return on investment, and if they don’t see it quickly, they move on. This isn’t a negative characteristic when you’re delivering production-ready software every two weeks or even more often.

I wrote one small note on my Kindle when I highlighted this quote, "definitely sounds like a flaw to me." This attitude struck me as having a cavalier disregard for deeper thought and more exploratory learning, especially for difficult problems. Maybe it works when you're implementing yet another CRUD app, but some software requires careful research and design to make something that's going to have a big impact. My interpretation of this statement is that I shouldn't be doing Agile Testing if I'm developing ground-breaking software, but I don't think that's really true. I think Agile promotes examining the trade-offs before making a decision on how to proceed, and sometimes the right decision is research and learning. AADD isn't really defensible.

Later in the book the authors talk about the difficulties of bringing new testers up to speed on a project, and they make this recommendation:

This conversation sparked the idea of a FAQ page, an outstanding issues list, or something along that line that would provide new testers a place to find all of the issues that had been identified but for which the decision had been made not to address them.

Ideas like this should be questioned as to whether or not they scale and if they add unnecessary complexity to the process with too many tools. I can't imagine such a thing scaling well at all, and anytime you create documentation that you'll never use personally, it will not be adequately maintained. I find that keeping these kinds of lists up-to-date is nearly impossible when they're not part of some other process that the maintainer does find useful.

The book did contain a few pleasantly surprising nuggets, though. Near the end when the authors were discussing defect tracking software, they claimed:

If your team is working closely with the programmers and is practicing pair testing as soon as a story is completed, we strongly recommend that you don’t log those bugs as long as the programmer addresses them right away.

It's an interesting difference from bug tracking software proponents, and such pragmatic practices can eliminate a lot of tedious bookkeeping.

All in all, I can't recommend this book either. I pretty much heard it all before and it has been presented much better in the other books I've already mentioned. Maybe if you're a tester that's completely new to the Agile process, you'd find Agile Testing much more useful, but I'd hope you could find a more concise resource. As for programmers, The Pragmatic Programmer and Robert Martin's books are more engaging and are all you're going to need.

More Than Enough Testing

I set out to fill what I thought was a gap in my understanding, and found out that other books I read had already covered what I needed to know. I probably didn't need to read either of these books. It was too much reading about testing and not enough practicing. You only need to read a little bit about testing to understand the benefits and how to go about it, and then it's best to spend the time experimenting with it. Clean Code and Agile Principles, Patterns, and Practices in C# are more than enough on the subject of testing for developers.

The Ivory Tower of Software Development and Messy Reality

What is the best way to learn something that is complex enough and broad enough that you will spend a lifetime mastering it? Some people believe the way to learn something is to first study the fundamentals and proper techniques, and only then move on to doing or creating things. Other people take a different approach and introduce new students to something simple, fun, and engaging right away to capture their interest and imagination before getting into the more technical aspects of the subject.

It's a question of a purist approach or a pragmatic approach—the ivory tower or the open market. This dichotomy has a strong presence in software development, but it exists almost everywhere.

Learning an Instrument

One example of this purist-pragmatist divide occurs when learning to play an instrument. Let's say you wanted to play the piano. You wouldn't have to go to lessons with too many different teachers before you would recognize different beliefs about teaching methods.

The purist approach would involve learning all of the proper techniques first. New student would learn scales, chords, finger position, and a bunch of other fundamentals, and then they would drill and drill and drill until they could do these things perfectly without thinking. They would do all kinds of exercises completely outside the context of real music, and then when they were ready, they could begin playing classical pieces. This is the ivory tower of perfection. Mistakes are eliminated quickly, bad habits are never allowed to take root, and the student is always on the straight and narrow path to musical virtuosity.

The pragmatic approach is a bit different. Instead of starting with technique, the students will start playing songs right away. The songs start out simple, to be sure, and proper technique is taught within the context of the songs that use those techniques. New skills are learned as needed to play new songs, and the focus is on playing real music. The belief is that the required skills will be learned in time; it doesn't have to all be done up front. Certain techniques may not be learned until much later and students may spend more time in areas that they enjoy. The students that want to become much better musicians and challenge themselves more will naturally put in the extra effort to master the more difficult techniques because the music is driving them to do it.

Which method would you prefer? They both have their merits, but I think more people are going to stick with the pragmatic approach because it's more fun and engaging. The purist approach quickly gets boring, and only the most dedicated (or most externally forced) students are going to get to the point where they can play actual music. Most people would give up on it before they got that far.

Programming Purity

Any sufficiently complex field will have this debate between the purists and the pragmatists about what is the best way to teach beginners coming into the field. The analog to the musical purist approach in the software domain is to have students learn all of the proper computer science theory first, before allowing them to write any real software. To a purist, the right computer science education would start with data structures and algorithms, preferably in Haskell or LISP (or both). It would quickly get into compiler theory and lambda calculus so that students would have a firm understanding of the most important parts of computer science.

I have no problem with languages like Haskell or LISP. I think they're great languages, and you'll learn a lot of stuff about programming by learning these languages. But as introductory languages for beginner programmers, they're terribly difficult and a huge barrier to entry.

The purist may claim that difficulty is a desirable trait in a language, and that one of the purposes of introductory programming courses should be to weed out students that don't understand these all important language features and computer science concepts. A computer science education should certify that a professional programmer knows how to write a compiler and an operating system, is proficient in the use of strongly typed languages, and has a firm understanding of closures and first class functions.

Taking this line of thought to its logical conclusion is absurd. Do we really want to require aspiring programmers to learn all the theory first, to certify professional programmers against some arbitrary set of standards, and to only allow professional programmers to write software? What would we test for? How would we differentiate between programmers that are allowed to write compilers from those that are allowed to write payroll software for an HR department? How would we establish which level of the ivory tower each programmer could work in?

Not every programmer needs to know the same stuff to be effective at what they do, and the vast majority of programmers out there don't need to know pure computer science theory to do their work. Besides, thinking that it's even possible to structure a perfect learning path for programming and force every aspiring programmer to follow it is just silly. People don't learn how to program that way.

How People Really Learn to Program

How did you learn how to program? I bet you didn't start by studying memory safety and resource allocation. I've never heard anyone describe their early programming experiences that way. I started out programming in LOGO, making the turtle draw simple shapes on the computer screen. Then I started writing clones of simple games in QBasic. It was fun and exciting, and that is what hooked me in. I wanted to learn more because these early programming experiences were fascinating. I hear a lot of stories similar to mine, but never have I heard a programmer say they got hooked on programming the first time they learned about closures or type safety.

People become programmers for a wide variety of reasons. Not every programmer loves programming, but the best programmers almost certainly do. The desire to program is necessary for learning the more advanced topics of programming. It's too hard otherwise, and the programmers that don't love it are going to find ways around having to think about the hard topics. In the end, that's okay. They'll still have plenty of work to do without the hard stuff.

For those that do love programming, they will be driven to keep getting better, and as a result they will learn the proper techniques and best methods of programming. The fundamentals are hugely important, and one of the best ways to get better is to practice and develop a deep understanding of the fundamentals. Like the piano student that learns the necessary techniques while playing beautiful music, the best programmers will learn the fundamentals while making great software. If they need to learn a certain programming concept to make their software better or to become more effective as a programmer, they'll do it, but the desire to program originated in those early experiences of writing simple programs to do cool things with a computer.

Creating cool programs in an accessible way should be the focus of any introduction to programming. It may be messy and impure, offending the sensibilities of the proprietors of the ivory tower, but it captures the imagination. And the imagination is a powerful force for learning complex subjects and doing awesome things.

The Benefits of Working on a Personal Software Project

If you love programming, then you probably already have one or more of your own software projects in the works. Personally, I find that working on a software project of significant size is one of the best ways to learn new tricks as a software engineer, but that is only one of many reasons to write software in your free time.

Choices

Before getting into the benefits of personal software projects, it's worth taking a brief look at some of the different kinds of projects you could explore. If you wanted to look at embedded programming, you could get yourself a great embedded system for less than $60 (sometimes much less). The most popular choices are Arduino and Raspberry Pi, but there is also the very powerful BeagleBoard and more specialized embedded boards like the TI LaunchPad series.

If you want to get into mobile development, all you need is a smart phone or an iPod Touch and you can do Android or iOS development. You could write a game, a productivity app made just for you, or satisfy some other nagging itch that's been plaguing you.

If you want to explore web development, you have tons of resources available, both online and off, for any number of web development frameworks: Ruby on Rails, Django, ASP.NET MVC, AngularJS, MeteorJS, or something newer and less well-known. There is no shortage of options, and you're bound to come across something new and exciting.

You could even do classic application development in Visual Studio with C# or XCode with Objective-C. There's also system-level programming in Linux. You could study up on your algorithms, look into machine learning, experiment with a new library, or explore what's new in big data. You can even dabble in parallel computing by programming your graphics card with CUDA.

The possibilities are nearly limitless and readily accessible at close to no cost. All of these types of projects are immensely fascinating, but to make any significant progress, you'll have to focus on only one and resist the urge to jump from one to the next like a raccoon chasing shiny objects. If you manage this tremendous feat and settle down to work on a project for an extended period of time, here are some of the benefits that you may experience.

Learn Something New

Starting with the obvious, if you pick a project in an area that you have little experience in, you're going to learn a ton of stuff. By working on a project that's not trivial, you will learn much more than if you had only read about it. Don't get me wrong, I love reading, and I read everything I can. But you won't learn a new language or pick up new skills simply by reading about it. Reading is hugely important and a necessary part of learning, but without something more to cement that knowledge in your brain, you'll quickly forget whatever you read. A project is a great learning tool because you'll struggle with things that seemed simple in the literature, and that struggle and the solutions that you discover along the way will stay with you much longer than what you read.

Practice, Practice, Practice

Projects don't have to only be about new things. You can do a project in a language and framework that you already know well and get some great practice out of it. You could get practice in other ways, like doing programming problems on various websites, but these problems focus on narrow pieces of a very large puzzle. While solving small, contrived problems can be fun and instructive—I thoroughly enjoy it myself—working on a larger software project will give you more practice in the context of something real. You'll also be developing other skills that small problems don't address, like designing the software architecture, deciding on the features you want to include or not, and defining the problems that need to be solved in the first place. Software development is so much more than merely writing code, and doing a real project will give you good practice at all of those non-coding skills.

Take New Tools for a Spin

For most people, using the hot, new programming language or framework at work is strictly forbidden. Unless you're working on a greenfield project in a company that's willing to take a risk on something that's probably not production ready, you'll be working in one of the well-established, mature programming environments at your day job. A personal software project will give you the opportunity to try out an intriguing new language that you've been meaning to learn, but has been off limits for your work projects. Who knows, if you get good enough at it, you could spin up some little utility projects at work to solve immediate problems that you're having and show off the new tool's advantages. One thing could lead to another, and you could end up adding real value to your company's business while getting a chance to play with new tools more often.

A Vehicle for Experimentation

When you've learned a new programming technique or a new language feature, it's probably best to not turn around and try your new-found knowledge out in a production code base right away. You're more likely to tick off your coworkers with all of the mistakes you'll make implementing a design pattern for the first time than actually adding value to a company project. Instead, try experimenting with new techniques on a personal project where you can turn your code upside down and inside out until you've gained a true understanding of the issues involved. Personal projects are also great sandboxes for trying out crazy ideas, just to see if they work.

Portfolio Gold Plating

A substantial personal project is a great addition to a work portfolio. If you've made some cool projects while tinkering at home, it's going to give you much more interesting stuff to talk about in an interview, and it's likely to help your CV stand out in a crowded field because it shows that you have a passion for programming. What's more interesting, saying that you implemented some of the business logic for yet another enterprise payroll system, or that you implemented an embedded control system for a robot that can navigate a maze? If the only kind of work experience you've been getting is the former, a personal project is your best opportunity to show off what awesome stuff you can do. Of course, doing a project for the sake of your portfolio alone won't work because the motivation won't be there, but if you're already doing it because you can't stop yourself, putting it in your portfolio is a bonus.

Glory and Honor

If you manage to build something in your spare time that's truly valuable and noteworthy, and you release it as OSS, you could make a real difference in other programmers' lives. A lot of the frameworks and libraries we use on a daily basis were made in programmers' spare time, and the authors are pretty well-known in the software development community. Fame can be a blessing and a curse, though, and these OSS projects can take on a life of their own. Persevering with a big open source project is a huge commitment, and it will take a lot of energy to see it through. Even more so than portfolio gold plating, you have to have the motivation to succeed with OSS or you won't get very far.

For Fun

This is the reason every passionate programmer does their own software projects, because it is great fun. Exploring new corners of the programming world, experimenting with esoteric and novel programming languages, creating useful things with your imagination and a computer—these are the reasons why programmers get excited about programming and play around with pet projects until the wee hours of the morning. Of all the benefits listed here, none is as important as having fun.

…And Profit

A personal project could eventually grow into a personal business or something more. Plenty of thriving businesses started out as a simple idea and some after hours effort. The amount of effort required to make a viable business is substantially more than messing around with pet projects, but if you think you have a good product that people would be willing to pay for, don't be afraid to charge for it. The worst that could happen is no one buys it. Then you've still got a great project for your portfolio, you learned a ton of stuff, and you hopefully had fun doing it.

It Needs to be Done

Sometimes there's a hole where a software product should be, and if you want it filled, you are the one who's going to have to do it. There's something about your life that keeps bugging you, and you simply can't find the right tool to fix it. You've searched endlessly on the app stores and Google and you've come up empty. Sometimes you don't even need to go to those lengths. Sometimes your problem is so narrow and personal that you know it's only going to get done right if you do it yourself. A custom piece of software can be one of the most satisfying projects because you're solving your own problem with your own skills and ideas. Nothing feels as great as that.

For the Children

If you have kids, working on a project at home can be a great learning experience for everyone. You get to teach them something that you love, and they get to spend time with one of their favorite people while doing something fascinating. Depending on their age, you'll have to tailor the project to something they can handle, but all kids love to build things, create things, and control things. Software projects cater to all of those desires, and it's great for them to learn how to solve problems through their own creativity and imagination. They'll learn how to overcome problems by your example. Watching you struggle with the problems that crop up in programming, realizing that they are solvable, and working through them together is invaluable. Watching their reactions when something finally starts working is incredibly fun and rewarding. The sheer force of excitement from seeing their creations come to life will keep both you and and your kids coming back for more.

Change the World

Big things start from small beginnings. If your personal project is filling a need that extends beyond yourself, it's quite possible that you could change the world. Change doesn't necessarily mean revolution, and the world doesn't necessarily mean all of planet Earth, at least in the beginning. Something that started out as a simple way to connect and share with other people or to optimize your health and lifestyle could grow into The Next Big Thing. Who knows? You'll never find out unless you start that project. Go learn something, have some fun, and change the world.

There Must Be A Better Way

As developers, we often try to defend our over-engineered and complicated solutions, and indeed, every problem has some amount of irreducible complexity. But there is often a simpler solution if we take the time to look for it. Part of good software development is believing in and spending the time to look for that simpler way. Don't resort to overly complex implementations just because the problem seems to call for it. The simple solution may not seem so shiny, but there is elegance in simplicity. It may not feel like you're practicing good engineering when you come up with a solution that looks too easy, but often times the simple solution is more robust and resistant to buggy edge cases.

In engineering and the sciences, simple is normally equated with elegant. A simple solution is easier to understand, maintain, and extend than a complicated one. The difficulty comes in finding a simple solution. It's hard work to understand a problem so thoroughly that the solution is made to look easy and obvious, but that is the mark of a good solution. All of that hard work may not be apparent in the end result, but it provides much more value than a complicated mess of code that people look at in awe because they can't believe that it even works.

Sometimes it seems like the problem is too complicated to warrant a simple solution. Other times the immense architecture of the complicated solution is a siren call that lures you in and blinds you to simpler alternatives. Finally, there are times when you stare in disbelief at the simplicity of the solution you came up with and convince yourself that such elegance isn't possible. Instead, you discard the easy option and pursue a more complicated path. I experience at least one of these traps with every engineering problem I work on, and it can be difficult to avoid them. Here are some things that I've found helpful in the quest for a better way.

Resist Over-engineering

Some programmers seem to revel in the intricate architectures that they create, claiming that the problem is so complex that a simpler solution is not possible. But what they've really created is an unmaintainable mess—a monstrosity that will collapse under its own weight before it ever gets close to shipping. You look at this pile of code and think, "there must be a better way," but the original architect can't see it because he's locked into the path he took and believes that the extra complexity adds necessary flexibility or extensibility.

Finding the simpler solution requires a completely different perspective and a willingness to solve only the problem at hand. A solution should be only as complex as the problem warrants. If the problem actually needs the added flexibility of the more complex solution, then it may be appropriate, but flexibility shouldn't be added for its own sake. Joel Spolsky railed against the unnecessary complexity touted by architecture astronauts when he praised pragmatic programmers that reject complicated frameworks and instead get stuff done. His advice is more pertinent than ever.

The basic idea is that design patterns, threading, templates, and other advanced programming techniques are complicated tools that should only be used when it is absolutely necessary. Don't use them just because you can. Use them when not using them will end up being more complicated. Deciding when to use the Factory pattern is a prime example of this concept. In theory, you could use the Factory pattern everywhere you instantiate an object, but that would clearly be overkill. On the other hand, situations arise where this pattern is genuinely useful. If you are instantiating similar objects from a class hierarchy in many places in your code, the Factory pattern will likely simplify your code instead of complicating it. That's when you want to use it.

I've been playing around on Exercism.io lately, and I've seen plenty of interesting examples of over-engineering in the solutions that programmers have submitted there. I thought about showing a couple examples here, but I don't want to single anyone out. Suffice it to say, some programmers submit verbose solutions with multi-level class hierarchies or delegation for simple problems like RNA transcription or converting decimals to Roman numerals. (Examples like these are plentiful on any coding site. Codewars.com and Project Euler are two more programming problem sites that allow you to compare your solutions with other programmers, and over-engineering is easy to find.)

If these programmers write such complicated code for simple problems, what do they do when the complexity of the problem scales up? I can only imagine. Complicated architectures have mental costs, and we only have so much brainpower to work with when programming. Why waste any of it on the overhead of an over-engineered, over-architected solution that solves more than the problem requires? It doesn't add clarity, and every time you have to deal with that code, it drains your mental reserves.

Find the Simple in the Complicated

Even after resisting the urge to over-engineer, the simple solution may still be elusive. Once the code is working, it will still be possible to make it better. I have not written or read a first-cut of a piece of code yet that couldn't be improved. That doesn't mean that every piece of code needs to be optimized, but if a part of the program needs to be extended or has issues that need to be resolved, finding a simpler solution is almost certainly an option.

I'm constantly amazed by the simplifications I miss when programming. While I didn't feel comfortable showcasing other programmers' code, I can certainly use my own, so here's my solution to the RNA Transcription problem on Exercism.

class Complement
  TO_RNA = {C:'G',G:'C',T:'A',A:'U'}
  TO_DNA = {C:'G',G:'C',U:'A',A:'T'}

  def self.of_dna string
    transcribe string, TO_RNA
  end

  def self.of_rna string
    transcribe string, TO_DNA
  end

  def self.transcribe string, transcription
    string.each_char.map { |c| transcription[c.to_sym] }.join
  end
end

The problem is fairly simple. It asks you to create a couple methods that return the RNA sequence for a DNA sequence (Compliment.of_dna) and vic versa (Compliment.of_rna). I thought this solution was pretty straightforward. I have named constant hashes for converting to RNA and to DNA, I use a generic transcribe method for both of the conversion methods, and the two conversion methods even read nicely, almost like English.

I was happy with the solution, but upon reviewing other submissions, I found that Ruby actually has exactly the method that I created as self.transcribe already included as part of the String class. I could simplify my solution quite a bit by using the String#tr method:

class Complement
  RNA = 'CGAU'
  DNA = 'GCTA'

  def self.of_dna string
    string.tr DNA, RNA
  end

  def self.of_rna string
    string.tr RNA, DNA
  end
end

Using the built-in method shaved four lines off the code and simplified the RNA and DNA constants. In the process of reviewing other people's code and striving to simplify my own, I learned about a Ruby String method that I hadn't known before and had some good practice expressing the essence of a particular problem. It's a great skill to cultivate.

Nearly all of the other problems I've solved have followed this same pattern. I try to come up with as simple of a solution as I can that directly expresses the core of the problem, and then I learn an even better way of doing it by reading other people's code. The same principle holds true for any programming that I do. If I need to find a better way of doing something, I can be sure that it exists; I just have to find it. The combination of Google and reading good programming books helps in the search for the problems I deal with outside of coding practice sites.

Don't Worry That the Simple Isn't Complicated

It may not feel like doing real work when you come up with a simple solution. You may feel guilty for producing something so obvious after a large investment of time! That's normal. Simple solutions are not something to be avoided because our egos believe that the problem warrants something more complicated. Don't confuse simple with simplistic. Simple solutions are more elegant, and are often more robust to edge cases and changing requirements than the overly-complicated alternatives. Simplistic solutions don't address all of the requirements and fall short of the problem's inherent complexity. We want simple solutions to complex problems.

If the simple solution works, don't worry that it looks small compared to the architectural edifice of a multiple-inheritance hierarchy. Coming back to Joel, he doesn't mince words on this issue:

You see, everybody else is too afraid of looking stupid because they just can’t keep enough facts in their head at once to make multiple inheritance, or templates, or COM, or multithreading, or any of that stuff work. So they sheepishly go along with whatever faddish programming craziness has come down from the architecture astronauts who speak at conferences and write books and articles and are so much smarter than us that they don’t realize that the stuff that they’re promoting is too hard for us.

Don't go along with the architecture astronauts if a simple solution will suffice. They may appear smarter, but they're also so deep in the complexity that they've created that they can't see that it will risk wasting everyone's time. A simple solution that works now and ships now is infinitely better than a complicated solution that will handle every possible future use case or additional feature but never ships.

Besides, complicated solutions always make me suspicious. A simple solution is a sign that the the problem is well-understood and the developers have thought hard about taking the right approach without generating any waste. Great programmers are like great teachers who really understand their subject and can teach it with a clarity of thought that makes complex material look easy. Great programmers will not complicate a design for the sake of over-engineering. They will search for the right solution that is appropriately simple and directly solves the problem. They don't worry about impressing others with complicated monstrosities when working software will prove their worth.

To be a great programmer, start believing that there must be a better way. Then find it.