10 Ways That Programming is Like Playing the Guitar

Connections between seemingly unrelated things make powerful learning devices. Those connections form links between things you are learning and things you already know, helping you to internalize new concepts. They also form stronger interconnections between different things already stored in your memory, and the cross pollination can give you a deeper understanding of both things. I'm always looking for ways to relate what I'm trying to master to what I already know, so when I began learning to play the guitar a couple years ago, I noticed a lot of similarities between it and programming. Here are ten ways that they are related.

You have to practice to get better.

Obviously. No one thinks you can learn the guitar by watching other people play or by listening to great guitar music on your iPod without actually picking up a guitar yourself and playing it. To get good, you have to practice a lot. Your fingers, hands, and arms have a lot to learn for even the simplest songs: fingering chords, picking strings, strumming, and learning basic form. Without practice, you could never learn or get better at playing the guitar.

The same is true of programming, yet most programmers don't practice. College coursework or what you do at your day job probably doesn't count in most cases. That's more equivalent to performing, not practicing. Practice involves dedicating time to learning new aspects of programming or working on those skills that are lagging. You can't do that while you're performing.

You get better faster if you practice consistently.

If you practice the guitar sporadically, only picking it up once a week or less, it is extremely difficult to get any better. You spend your whole practice trying to get back to where you were the last time you played, and you have no time to work on anything new. On the other hand, if you practice 5-7 days a week for an hour or two at a time, you're going to improve rapidly. If you do that practice session at the same time and in the same setting everyday, you'll do even better because your body and mind will trigger on the familiar environment and switch right into practice mode. You can whip through a quick warm-up and get into the meat of your practice without any effort. You know what to do and you don't waste any time getting it done.


You can get the same benefit from consistency with your programming practice. Do it in the same place with the same programming environment and you'll get right into the groove every time. Try practicing everyday at the same time to make it a routine.

You get better faster if you push yourself.

You'll plateau fairly quickly if you learn to play a couple songs and then only practice those songs over and over. It may feel comfortable to be able to play some songs well, but you have to get out of your comfort zone and learn to play new stuff. It's frustrating to practice hard pieces that you haven't memorized yet, or that have complicated passages that tie your fingers in knots. But you won't get better by only practicing what you know. You have to push into uncharted territory to grow.

Practices should consist of about 80% music that you know how to play and 20% music that's just beyond your current abilities. Any more hard stuff than that and the practices can get too frustrating; any less and you won't be progressing as fast as you could. And don't shy away from the hardest part of a piece. That is the part that you should be focusing on most. Sometimes it's a few especially fast measures or a few oddly spaced notes or a tricky chord transition that's causing the most trouble. Play through that particular trouble section. Then reset and play it again and again and again until you've engrained the motion into your muscle memory and can play it without thinking. Then play through the entire song and see if the same measures come right out without any effort. If not, then go back and practice some more. Sometimes you won't get it in one practice session. Try repeating the problem area a set number of times each session, and eventually the right motion will sink in, and you can move onto the next problem area.

Programming has potential problem areas, too. What parts of the language or libraries you're using are difficult to understand? What features are hard to remember or use effectively? What areas of programming do you avoid because you haven't put in the time to learn them? These are the areas where you should be pushing yourself.

The best musicians always practice the fundamentals.

My guitar teacher told me a story of a journalist who was attending a concert featuring a famous pianist. Unfortunately I cannot remember either of their names now. Anyway, the journalist was going to interview this pianist the night before the concert, but when he went to his hotel room, he heard the pianist practicing and decided to listen outside the door for awhile. He would hear a single chord played, then a few second pause, then another chord, and another pause. This went on for some time before the journalist realized that the chords actually were part of one of the pieces that would be played the next day. This famous pianist, one of the best in the world, was practicing his concert pieces one chord at a time, making sure that every key press was absolutely perfect. Even the night before a concert, he was practicing the fundamentals in intimate detail.

Now in that case it was a pianist, but I'm sure that the best guitar players practice the fundamentals just as intensely. Every chord fingering, strum, pick, arpeggio, slide, and hammer on is practiced until it can be played without thinking. The fingers know what to do on their own so that the mind can be free to think about higher level tasks such as composition and style.

What would be the fundamentals to practice in programming? Language constructs, data structures, and algorithms should all be known inside and out. If you know these things cold, you can spend your mental energy on the higher level tasks of software engineering. They are the tools that make up your programming toolbox, and the more tools you know how to use, the more you can do. There are plenty of ways to practice as well. You can go through problems in your favorite algorithms book, implementing them in the language you're practicing. You can get ideas from websites that specialize in practicing programming. You can make up your own exercises and flex your creative muscles while your practicing. The more you practice the fundamentals, the better you're going to get.

The best musicians make hard pieces look easy.

It is amazing to watch a great guitar player. They make even the most complicated pieces look effortless. They have played for so long and practiced with such dedication that their instrument seems like an extension of themselves. They can do things with a guitar that most people wouldn't think possible. The best programmers can do the same thing with their instruments. They have spent so much time learning their tools that they know their text editors, their programming languages, and their environments inside out. And they can make them sing.

The more you practice, the easier it gets.

One thing I started noticing after months of guitar practice was that I would improve without realizing it. After learning a new song, I would attempt to play along with the recording, trying my best to keep up, but failing completely. I would practice the song for days or weeks, feeling like I wasn't making any progress. Then I would try playing along with the recording again and surprisingly, find that I was playing too fast. The improvements snuck in under my radar. I wasn't able to see the progress I was making without some external baseline to compare against. My wife would often provide another baseline. Periodically while practicing a new song, she would say, "Hey, I know what song you're playing!" That's always a good sign.

The point is, as long as you are practicing the fundamentals and the hard parts and doing it consistently, you are getting better, even if it doesn't feel like it. Eventually you'll reach a point where you can do things with ease that used to be a constant struggle. In programming, don't shy away from the difficult concepts. Attack them relentlessly until you can use them effortlessly.

Start by copying from the best.

When starting out with guitar practice, I picked songs that I knew well and wanted to learn how to play myself. Since these were some of my favorite songs with strong guitar parts, I had a clear idea of how to reproduce them, I easily noticed mistakes when I made them, and I could tell I was playing them right when they sounded like the song in my head. As I get better, I try out more difficult songs and learn new techniques. All of this becomes part of my growing repertoire. At some point I'll be able to create my own music and learn improvisation, but the best way for me to progress right now is to imitate and learn from the best.

The same concept should be applied in programming. Seek out good, clean source code to read and learn from. There is a wealth of open source code out there, and you likely have a favorite program that you can get the source for. Read through it and try to understand how the programmers structured the code. What design patterns do they use? What is their coding style? How do they use the language to accomplish their goals. Good source code contains a wealth of knowledge for you to explore.

Music is about patterns and sequences.

A song is not made up of one long series of individual notes. If it was, it would be almost impossible to memorize. A song is made up of sequences of notes, called phrases, that are equivalent to code blocks in programming. There are repeating patterns similar to program loops. There are codas that act like code branches or functions. Many times phrases are repeated with slight variations or shifts up or down the scale to create larger phrases from similar smaller ones.

The guitar chords are full of patterns, too. Most chords are created from only a few basic finger formations that are slid up or down the frets or back and forth to different sets of strings. Sequences of chords tend to follow patterns, and combining them with different strumming patterns creates all kinds of rich, rhythmic sounds. Arpeggios are chords that are played by quickly picking the strings in a pattern instead of strumming. All of these different patterns and sequences, once learned, can be strung together to create very complex sounding music while still being relatively easy to remember.

Programming is quite similar with basic code blocks being built up to create massively complex systems. Without the patterns and sequences that we have developed in programming, code would be a garbled mess, and we couldn't possibly hope to create the software systems we have today.

When learning a new piece, break it into small pieces.

Knowledge that the structure of music is made up of simpler patterns and sequences offers a great way to learn new pieces. If you know the patterns and how they repeat, you can easily play along with a song after only a few practice sessions. I've noticed this ability improving as I learn more chords and techniques. Chord sequences that I learn from one song show up in other songs all the time, and the more I learn, the faster I pick up new songs. Practicing sequences greatly improves muscle memory as well. This is how great guitar players can play a new song after having only heard it once, or even jump in and play along after hearing only a few measures.

The parallels to programming should be fairly obvious by now - picking up new songs is like writing new programs - but the connection goes beyond that. Once you know the basic patterns and sequences of programming, it becomes progressively easier to learn new programming languages. Most knowledge in one language is transferable to others, and the more languages you know, the better programmer you become.

The Zone feels good.

When you get into The Zone, it's as if the world melts away and the only thing left is you, your instrument, and the music. Time stands still. Your fingers dance across the strings. The notes resonate with your soul. These are the moments that are most satisfying. At first it doesn't happen often, but the better you get, the more time you spend in The Zone. Programming is much the same.

Beyond the Task List

I used to be pretty big on task lists. It's such a highly touted way to stay organized, and what could be simpler? List all of the tasks you need to do for all of the projects you have to do, and then check them off as you complete each one. Before you know it, the list is all crossed off and you can bask in all that you have accomplished, right?

The basic task list has many variations. The first one you were likely introduced to was the assignment notebook to keep track of all of your homework and when it was due. It was promoted as an invaluable time management tool that would allow you to juggle the multitude of assignments that you had to deal with every week. Then there's the social calendar, which comes in a slightly different grid form, and tasks - I mean events - get checked off as the days pass. If you have a better half, one of the two of you will have to take primary ownership of the social calendar, or be prepared for double bookings, conflicts, and general confusion. Then, of course, there is the honey-do list, filled with hopes and dreams and expectations just waiting to be... waiting to... well, you get the idea.

Entire systems have been built around the task list. Any email client of reasonable size has a task list and calendar that link with the inbox to make task generation as easy as clicking a button. I've used Outlook's and Thunderbird's task list features in the past. Most note-taking software also includes task list features, and then there are programs whose sole purpose is task management. There's even one that's a game. You can buy any number of paper-based systems that come in notebooks or binders with all kinds of dividers and tabs and other organizational tricks to make a polished system that promises to simplify your life. You can even take training courses in these systems to learn how to fully utilize all of the organizing power they have to offer.

A Solution in Search of a Problem

At one time I tried one of these systems myself, the FranklinCovey Planner. I never took one of their courses, but I did talk my wife into using the binder planners with me. We are still using parts of the planners today. We cut out all of the unused pages and are using them for scrap paper to jot down notes and grocery lists. Very useful, but a plain old notepad would have been a bit cheaper. You see, there are so many problems with task lists and their brethren that most people will not stick with them for any length of time. I never could. Here are the problems I can think of off the top of my head:

• Task lists are a time waster. Small, targeted task lists that focus on a specific goal are fine. Making a quick task list to keep a bunch of little errands straight can be quite helpful in a pinch. But if you are making comprehensive task lists to manage every aspect of your day to day life, you're probably spending too much time making lists and not enough time getting stuff done. You should be spending that time on more important pursuits.
• A task list is (almost) never finished. I have rarely, if ever, completed a task list that was more than about five items long or requires more than a couple of days to finish. The real problem is that I add items to the list that I think should be done for completion's sake, but I honestly never intend to do. Then I give up on the list before checking off the last items so I miss the most satisfying part of using a task list.
• A task list is never up-to-date. If the completion of the task list is going to take more than a couple days, the action items will change during the course of the project. Some items on the list will become irrelevant. Other items not on the list will become necessary. Pretty soon you are spending too much time maintaining the list instead of making progress toward your goals.
• You are contorting your life to a list. Life is not like a task list. We are not robots that follow nicely ordered, linear procedures, and we shouldn't try to be that way. Real life is fluid, nonlinear, and chaotic. We have to deal with obstacles, make branching decisions, and generally react to diverse conditions. Trying to shoehorn your activities into a task list is a fool's errand.
• Task lists are not dynamic. Just as tasks need to be added and removed from the list while a project is in process, pretty much everything else about the tasks will change during the project. The relative priorities of tasks, the execution of tasks, the meaning of tasks, and the relation of the tasks to other ongoing projects will all change over time. The completion of certain tasks will affect other tasks in unforeseen ways, resulting in more work managing the task list instead of making real progress.
• You don't need a task list. If you have any experience with the activity you are doing, you already know the next couple of things you have to do to make progress. Pick the one thing that will allow you to make the most progress, and do that thing next. Repeat until done. If you don't have the experience, then you're not going to be able to create a viable task list anyway, so why waste your time?
• Task lists are a burden. Every time I've tried making and maintaining a general task list, I've fallen into the mindset of finishing the list before adding anything new to it. I actively resist taking on more action items because I want to see that list getting shorter, not longer. I don't have much trouble removing tasks that become irrelevant, but that is a small subset of the necessary changes that a task list has to endure to reflect reality. Having this completion mindset exacerbates most of the problems I've described here.

There Has Got to be a Better Way

I've found that in most cases, I don't really need a task list. What I need is a list of choices from which I can pick the best option to work on next that will provide the biggest gain for time spent. That list of choices can take many forms, and I try to use the simplest form that will work for the goal I'm working toward. Sometimes it's useful to categorize or tag items as the list grows. Sometimes different sorting options and capabilities are important. Sometimes the list needs to be extremely flexible, and it ends up not looking much like a list at all.

The big advantage of a list of choices over a task list is the change in mindset. Instead of a rigid list of things that need to be completed and checked off in lock-step fashion, you have a fluid list of options that can grow and contract with your current needs without the mental stress of completion anxiety. Here are some examples of lists of choices that I'm currently using.

• Life Mastery List - This list should be short enough to fit in your memory without any assistance because you should always have this list with you. Whenever you have some spare time, you could work on something from this list no matter where you are. At least one thing from this list will require nothing more than your own thoughts and possibly a pen and paper. Sometimes you'll have an item on this list that you can work on with a little ingenuity, like juggling. You can almost always find something to juggle.
• Amazon Wishlists - These lists are great for holding all of the media that you want to explore. I have hundreds of books, movies, video games, sheet music and documentaries in over a dozen different lists. I know that I'm never going to finish these lists because they keep growing and I'm always adding new ones. But that's okay because I don't expect to finish them. I use them to tag interesting things that I may want to look at later. If a list gets too big, I break it up into smaller lists. Then when I have time for a new book or game, I can peruse the lists and pick out something to do without having to remember what I had thought was worth my time six months ago.
• Google Drive - Both the document and spreadsheet are great low-tech tools for keeping track of ideas. Spreadsheets are good for things that have defined properties that can be sorted. Documents are better for ideas that need descriptions, like my list of ideas for blog posts. Best of all, they're available for me wherever I have an internet connection, and search is built in. Having the ability to search across lists makes the lists much more dynamic and flexible without complicating the interface.
• Trello - This is a web app that allows you to make lists of cards on a board. Each card has a short description of a task or idea, and the card can be expanded with checklists, due dates, comments, and all kinds of other things. You can have multiple boards, and share boards among users for collaborative projects. It's great for keeping track of projects at work, and it's designed for constant change. Check it out!

All of these tools help me to stay organized and remember the important ideas and tasks without being overwhelming. They are the simplest tools necessary to get the job done, with as little overhead as possible. I spend a very minimal amount of time in these tools, but they are almost always available when I need them. It only takes marginally more time to add an idea to one of these tools than thinking of the idea to begin with, and if I can't get them right away, I'll write the idea down and add it later.

The Task List is Finished

Other than the occasional weekend to-do list, I no longer have much use for task lists. The social calendar still works quite well, but that's a special case of needing something to keep track of appointments, obligations, and social events. In most other cases, I've found that there are much better tools for getting things done. Having a list that provides you with options instead of regimented tasks enables better control of the process. It will free up time to make progress and lead to much more satisfaction. It's time to stop messing with task lists and start getting things done.

Putting in the Requisite Time

When I was doing ASIC design at my previous company, we would routinely come up against bugs in our ICs that at first seemed unfathomable. Sometimes we would get a brand new tube of parts for a new design that was rigorously simulated, and after waiting for months for the first silicon to arrive, we would power up the parts and get behavior we couldn't even dream of seeing. Other times we would get returns from customers with failure modes that would leave us searching for our sanity in the lab. No really, on more than one occasion I was debugging a failure in the lab, someone would come in and ask how things were going, and the only way I could respond was to say, "I must be going crazy, because this cannot be happening."

No matter how strange a bug looked at the outset, I do believe we always got to the answer in the end. I cannot recall a single bug for any of our projects that we did not solve. Notice I didn't say "fixed." Not all bugs are fixed. Sometimes the cost-benefit tradeoff isn't enough to go ahead with the fix, or a workaround exists that adequately avoids the bug. Even in those cases, every known bug should be well understood. The team I worked on took bugs very seriously, and we never gave up when searching for the root cause of an elusive bug.

Lost in the Woods

Most bugs were easy to solve in a matter of hours or days. The symptoms were clear and reproducible. The underlying causes were simple to analyze and simulate to solve the problem. In some cases debugging would stretch out into weeks or, even more rarely, months. After solving a couple of those types of bugs, I started to appreciate that if you pursue it long enough, the answer will present itself. If not, you have not yet put in the requisite time. The answer is there, waiting to be discovered, but it's going to take a certain amount of time to find it. Eventually it became a saying. Whenever someone was getting discouraged on a bug-finding mission, we could remind them, "You just haven't put in the requisite time." And that would improve their resolve.

This idea is not destiny at work. It's more of a path through the woods that takes time to travel. Your starting point depends on the knowledge you have of the design. The path you take depends on the skills and experience you have in debugging. And the speed at which you go depends on the tools and resources you have available. When you actually find the root cause, it's going to be sudden. The trees will part and the solution will be revealed without any warning, so you better be paying attention. I've always found this to be true. Whether the bug is solved in hours or months, there is no slow buildup to the solution. Everything you've learned about the problem and every piece of data you've gathered culminates in the immediate realization of what's causing the bug. Often you look back and wonder why it took so long because the solution is so obvious in hindsight.

Perhaps more interesting is that this realization can happen anywhere at any time. I've had it happen while making breakfast, while walking to or from work, and especially before falling asleep. I had a coworker who routinely thought of solutions while in the shower or brushing his teeth. Everyone probably has a slightly different prime activity that allows them to think clearly, but more often than not that activity is not sitting in front of the problem, trying to solve it. Don't get me wrong, that time spent in front of the problem is valuable and necessary. You need to take measurements, gather data, and process information to learn as much as possible about the bug. But I've found that I do my best thinking when I can defocus and let my mind wander through all of the information about the problem that I have loaded in my head. I will often stumble upon the solution or a critical insight without realizing that I'm thinking seriously about the problem. Those moments are sublime.

The Ghost in the Machine

What do you do when you haven't been able to find the root cause of a bug for years? I've had one such experience so far, and hopefully it will be the only one. It was on one of the ASIC projects I worked on at my previous company. I'll keep the descriptions vague enough to protect the innocent, so to speak, but I'll try to give enough detail to keep it interesting and hopefully informative.

Let's call it The Bug, just to put it on a grander scale. It didn't actually take years of dedicated, focused effort to ferret out The Bug. I'm not sure how I would have survived an ordeal like that. I would have gone positively loopy looking for a bug for that long. But from the time the symptoms were first found to when the root cause was finally uncovered was at least three years, possibly as long as five. I'm a little fuzzy on that detail since this long trek was interspersed with numerous other projects and bugs, but at any rate, it was a long time.

The initial problem was that our customer found a part whose output would get stuck at a high voltage after it was powered on, with no response to changes in the input. Power cycle the part and it would behave normally.

Almost right away we had a suspicion that The Bug was not going to be easy to find. The first thing you do when dealing with a new bug is try to reliably reproduce it. Well, that alone proved difficult. The part would seem to get into its stuck state completely randomly and extremely rarely. That's a bad combination. Oh, and it would only happen when it was in a temperature range of 75-85 degrees Celsius. Worse. On top of that it was encased in a plastic epoxy so that the part and its surrounding circuit were completely inaccessible. Even worse!

It took us weeks just to figure out the optimal temperature to coax the part into sometimes getting into its stuck output state. Then we had to deal with the fact that we had no visibility into the part. All we had was three wires - power, ground, and the output. The output was also a communication line to the part, but getting into that mode required a reset that would clear the error. The input signals were accessible, but we quickly eliminated that as a possible source of the error, and we were back to square one.

We were dealing with two critical debugging issues here - reproducibility and visibility. If you can reproduce a bug on demand and you have perfect visibility into the system, that bug is going to be toast in short order. Not so for The Bug. We were basically flying blind in a random world. Not. A. Fun. Time. Luckily, the customer had found a few more parts that exhibited the same behavior. I say this in hindsight, of course. We didn't think it was so lucky at the time. Up until that point we had to be very careful with our only errant part. The extra failures gave us the leeway to be more aggressive with our debugging.

We had to increase our visibility, so we decided to try milling out some of that plastic epoxy to get at the pins of the ASIC and see what was going on there.  Miraculously, the first attempt was successful and we had a still-operational part that, most importantly, still had the output getting stuck. Unfortunately, we didn't see anything unusual at the ASIC pins. Even the reset pin was being released properly. Our best running theory at the time was the part was stuck in reset because the stuck output value was the same as if it was in reset. That stuck-in-reset-like behavior should have been a clue, but we weren't ready for that, yet. We hadn't put in the requisite time.

After a few weeks of messing around with ASIC pins and milling, we weren't able to find anything else noteworthy, so we decided to take the next step and expose the silicon die by decapping the ASIC. This step was another attempt to increase our visibility into the system. There were numerous complications because the epoxy couldn't be dissolved by the chemicals used to remove the plastic package, stronger chemicals might destroy the die, and the surrounding epoxy interfered with the whole process. It took a lot longer than usual, but after some amazing work by the failure analysis team at an external lab, we were able to get a still-working-and-failing part with an exposed die.

We did a bunch of cool tests on the die while it was in the stuck output state, and got lots of colorful close-up pictures of different parts of the circuit. But we still didn't find anything compelling. To increase visibility again, we decided to pull out the die and put it in a new package so we could analyze it separately from the rest of the system. The transplant was unsuccessful. Each one of these processing steps carried a risk of irrecoverable damage, and we had finally reached our luck limit. We would have to take a different route.

We decided to investigate some suspicious defects on the surface of the die, which was a destructive process. Since we had other failures to analyze, and this die was no longer working, we could afford to sacrifice it. What we found were places where it looked like the metal wires on the die surface were cracked and crushed. That kind of damage could be caused by particles in the packaging plastic that were too large. The interaction between the epoxy, the plastic package, and the die surface could cause some of those particles to push into the die surface, causing damage and erratic circuit behavior. This behavior could change with temperature as the materials involved expanded at different rates with temperature changes. It looked like we had our root cause.

Conveniently, we were in the process of changing packaging houses, so we started analyzing parts from the new packaging house, and we couldn't find any problems. Even though it is impossible to prove a negative, we were fairly convinced that we had solved The Bug, and we were ready to close it. I've oversimplified a great many things here, and this entire process had taken almost a year with all kinds of meetings, brainstorming, analysis, and testing. We had done a lot of failure analysis of the other parts that showed the stuck output as well, and we thought all the evidence pointed to the packaging material as the root cause.

When a Bug Becomes a Zombie

At this point I should mention two things about debugging. First, if you cannot turn the bug on and off, you do not truly understand the bug! If you understand a bug, you can recreate the conditions that make it show itself, and you can remove the conditions (or the bug) to make the system behave correctly. It's like a switch - bug on, bug off - and if you can't control it, you don't understand it. We were not able to do this with The Bug, and at the time we didn't think it was possible because of the root cause. It just wasn't possible to recreate that kind of packaging defect or correct it once it happened with any kind of reliability.

Second, debugging a system with low visibility is hard. ASICs generally fall into this category of systems regardless of whether you have access to the package pins or the die itself. Your measurement options are always limited, and if the bug is not within your line of sight into the circuit or you can't deduce its location from the view that you have, it's extremely difficult to find it. Simulations can be helpful because they provide much more visibility, but if the bug is related to startup or power conditions, simulations can get pretty inaccurate and unreliable. Debuggers are a thing of dreams in the world of ASICs. As we saw with the first attempt at solving The Bug, most of the effort revolved around increasing our limited visibility into an opaque system. With those two ideas in mind, let's continue our tale.

So the design team had moved on. There were other bugs to slay and other projects to finish. A year or two passed, and we thought we had The Bug behind us. Then one day we got a call from the customer with some startling news. They were seeing an unusually high failure rate on their production line with parts' outputs getting stuck at a high voltage and failing calibration. They were recoverable by cycling power. Uh-oh. The Bug had come back from the dead.

The customer had already done a fair amount of analysis, and discovered some incredibly useful things about these new failures. First, they were failing at room temperature, which would make our debugging process much easier. Second, the customer was able to deterministically reproduce the failure by quickly power cycling the failed parts from on to off to back on in a very short time. A bad part's output would get stuck high, guaranteed, which meant we could easily reproduce The Bug for measurements and know for sure if any applied conditions truly fixed The Bug. And third, they had caught a number of failures before injecting the plastic epoxy that had been the bane of our existence in the last go around with The Bug.

Suffice it to say, we were in a much better position to find The Bug this time. We fairly quickly characterized the behavior and got to work trying things out. One difficulty with debugging is that the longer it goes on, the less methodical it gets. You may start out with a defined, logical process of measuring the behavior of the bug, narrowing your search, and eliminating theories of the root cause. But at some point you run out of theories, and at that point you can't be afraid to start throwing things at the wall to see what sticks. When you enter this wild experimental theory mode, the faster you can generate ideas and try them out, the better. Anything goes. Don't discount off-the-wall ideas because the normal debugging process has failed, and the root cause is only going to be found by more radical means.

If you hadn't already guessed, we were approaching wild experimental theory mode. We had a few parts decapped since we had a decent collection of them, and we were doing the routine emission tests on the exposed die with no clear results. We were becoming more convinced that the problem was with the power-on-reset circuit because of the way the failed parts reacted to rapid power cycling, but all of our measurements showed that the chip reset was okay.

One day I finally decided to go in with some micro-probes and a microscope and actually measure the low-level signals feeding the power-on-reset circuit while power cycling the part. This was no easy feat with all of the lab equipment hooked up around the microscope and wires hanging off the probes to create the right conditions for reproducing the failure, but I got lucky. The wires I needed to probe on the ASIC surface passed through an open area of the die with no other signals around them, so it was relatively easy to set a probe down on them. What I found was that a critical reference voltage was not coming up fast enough on power on, so the reset that looked like it was released properly on power on was actually not getting asserted at all. What's more, I could force the reference voltage to come up faster and then the stuck output never happened. Bingo!

Always Do a Postmortem

The rest of the details fell into place immediately, but they aren't important. The Bug was revealed after years of hiding, and of course, it was now so obvious. We had been blinded by a number of assumptions that continually lead us astray. It was only after an extended hiatus and the reappearance of The Bug in an easier form, that we were prepared to find the real root cause. It was as if we were different people looking at The Bug anew.

Was it possible to have shortened the time necessary to find this bug? Linus Torvalds has a saying that "given enough eyeballs, all bugs are shallow." That certainly applies in the open source community where many individuals work on a bug independently, but I'm not so sure it applies in this case. We definitely tried involving lots of smart people to find The Bug, but beyond merely generating more ideas, which was helpful when the core team was hitting a wall, mostly what happened was group think that lead us to dead ends. The problem was that we couldn't easily work independently with such a limited number of failures, and only the core team had the knowledge of the design and use of the debugging tools to actively search for The Bug. If we could have solved these debug scaling problems, we likely could have made The Bug more shallow.

In hindsight, the stuck output that looked suspiciously like the output at reset should have been a giant clue, and we should have dug into that power-on-reset circuit more deeply from the beginning. This particular bug ended up requiring years of investigation and learning before we were ready to find it. I hope I learned enough in that experience to prevent a repeat performance in the future, but even if I do come across another monster bug like this, I'll be more prepared to put in the requisite time. Remember, given enough effort every bug's time will come.

Tech Book Face Off: Here Comes Everybody Vs. Cognitive Surplus

We are in the midst of a revolution. It's not the Internet Revolution. It's bigger than that. The internet is only the mechanism that is making it possible. The changes that are resulting from this revolution will be so profound and widespread that, even though we have been experiencing tremendous changes until now, we can only speculate and dream of what changes are still to come. So far these changes have been mostly limited to communication and social interaction, with some signs of an impact in education. Of course, the changes so far have already had a great effect on how we do business and live our daily lives, but they are still only the beginning. The principles of many-to-many communication and distributed sharing and cooperation are set to change every other aspect of our society, including transportation, energy, and government.

These are the ideas that have been buzzing around in my head since reading two books by Clay Shirky that lay out the reasons why the internet and social networks are having such a huge impact on our society.

VS.

Here Comes Everybody

How has the internet changed our lives, and why are those changes happening on such a large scale? Those are the questions that this book answers quite thoroughly and effectively. Clay runs through the logic of how the internet could be so fantastically disruptive to the status quo and how entrenched interests, such as journalism and media companies, were blindsided by the seemingly innocent reduction of publishing costs and increase in interconnectedness online. He shows his reasoning through a series of engaging stories and examples that clearly illustrate the power of sharing and connecting on a massive scale.

Not only did he do a great job of story telling, but he constantly had me thinking of the wider implications of his arguments.

"When old costs are shed, the time and money saved can be applied to new things, things that were unpredictable in the old regime."

The cost and effort of publication are now near zero, and the explosion of content online is the self-evident result. This change has had a permanent effect on all kinds of media providers, but especially news outlets and research publications. When established channels no longer have control over the flow of information because anyone can publish whatever they wish, new and potentially ground-breaking ideas have a much larger audience. Ideas that thwart conventional wisdom or run counter to mainstream acceptance have a much better chance of wider attention and adoption. The trick is to reach the surface of the growing ocean of noise and get noticed. Luckily, a lot of great ideas seem to have an uncanny ability to do just that when executed well.

"The distinction between communications and broadcast media was always a function of technology rather than a deep truth about human nature."

We are inherently social beings. We have always depended on the dissemination of knowledge, first for our survival and then for cultural and technological progress. Historically we were not capable of communicating outside of a small group of family and friends unless it was one-to-one connections via telephone that had no wider audience or one-to-many connections via broadcasting that had no immediate feedback. Suddenly we can have many-to-many connections that are bidirectional and nearly real-time. We are finding fascinating new uses for this capability everyday.

"Communication tools don't get socially interesting until they get technologically boring."

At first new technology is seen as an end in itself, and is normally priced beyond the reach of most people. As prices inevitably fall and adoption rates rise, the technology becomes a commodity and real change begins. Smart phones are the obvious example that shows the potential of communication tools connecting billions of people worldwide. Events of all kinds are now being captured, documented, and shared permanently for everyone to witness. The real consequence of this trend is that we are quickly approaching the full realization of the phrase "the truth will out." The information is out there. All we have to do is pay attention and leverage our ability to collectively filter out the noise.

"Open source is a profound threat, not because the open source ecosystem is outsucceeding commercial efforts but because it is outfailing them."

When failure is costly, we act to minimize failure and focus on risk aversion. With fixed costs experimentation is stifled by the ability to pay for only a few trials, at least one of which must be moderately successful to yield a profit. But when failure costs are effectively eliminated, we can afford to find the wildly obscure implementation that becomes wildly successful. To compete in such a world, you must first accept that you must increase your failure rate and focus on improving your awareness of successful trials as they are happening.

These are but a few of the excellent observations contained in this book, and I have only scratched the surface of Clay's analysis. You'll have to read it to get the rest because I need to move on to his next book.

Cognitive Surplus

Whereas Here Comes Everybody went through the mechanics of social networking tools, Cognitive Surplus addresses the unrealized potential for those tools to effect change within our society. Clay continues with his superb examples and storytelling, and allows the reader's imagination to run wild with the possibilities yet to come.

With so much extra time and effort being harnessed through online sharing, collaboration, and content creation, truly incredible tools and stores of knowledge are being created. Linux and Wikipedia are prominent examples that resulted from only a small fraction of that time and effort. What would be possible if more of that energy could be focused on solving some of the world's critical problems?

"The bigger the opportunity offered by new tools, the less completely anyone can extrapolate the future from the previous shape of society."

Clay doesn't have the answers, of course, but he does warn us not to underestimate the power of small contributions.

"The dramatically reduced cost of public address, and the dramatically increased size of the population wired together, means that we can now turn massive aggregations of small contributions into things of lasting value. This fact, key to our current era, has been a persistent surprise. At every turn, skeptical observers have attacked the idea that pooling our cognitive surplus could work to create anything worthwhile, or suggested that if it does work, it is a kind of cheating, because sharing at a scale that competes with older institutions is somehow wrong."

I completely agree with that assessment. If anything is certain, more established institutions will be reduced or eliminated through the revolution wrought by the internet. The power of traditional organizations to control the flow of information is quickly succumbing to the torrent of online contributions, and I think in the end, the individual will have a much stronger voice than before. If we are able to protect the independence of the internet, we will certainly all benefit.

There is so much more to Cognitive Surplus than what I have summarized here. Clay covers an amazing amount of ground, touching on so many critical aspects of our new socially connected environment. I highly recommend reading it to get the full effect.

Contemplating the Future

If you haven't already been convinced, I highly recommend both of these excellent books. Either one stands on their own, but together they tell a gripping story of how the internet has and will continue to change our lives, mostly for the better. If you are designing for the web, trying to create an online community, or are at all curious about how the internet is changing all the rules, you definitely need to read these books.

While it is fairly clear that the internet is drastically changing the media and social networking landscapes because of the shift in communication effort and cost, should we even consider that it will change any other major sectors? Well, research and education are already changing in major ways because of the free availability and easy access to information on the internet. Wikipedia alone has permanently altered the way we process and store the entire body of human knowledge, but that will certainly not be the end, only the beginning. As sites develop better organization and filtering of information so that it can be presented in more meaningful and understandable ways, the sharing and learning of knowledge will definitely accelerate. The current state of online education, especially, is only beginning to show the potential of what it could become given the power inherent in the web's interactivity.

How about transportation. We are already seeing websites that help you find carpool partners, do ridesharing, or even car sharing. It's fascinating to see that people are actually finding ways to make these things work relatively safely and cheaply. Some pretty crazy ideas are actually turning out to be fairly effective here. What else could we find that works to make commuting and travel more efficient? The new electric cars that are coming out can potentially benefit from the connectivity of the internet as well. The communities growing up around the Tesla cars and the Nissan LEAF are quite active and passionate, and if the auto manufacturers cultivate those communities well, they could capitalize on all those users' knowledge and experience with these new vehicles.

Then there's energy generation and distribution. Here is where things really get interesting because until now energy generation has primarily been centralized and then broadcast to consumers, much like television. It is starting to become apparent that this configuration is not going to last. There may be large solar and wind farms in the future. But I can easily see the model for energy mirroring that of the internet, with individuals generating, storing, and sharing their own solar or wind energy in a global energy grid. Such a distributed, shared grid would do for energy what the internet has done for communication and information, giving vastly increased freedom to individuals.

Speaking of vastly increased freedom, the internet has the potential to do the same for individuals with respect to government. So far, the recently much more interconnected citizenry has made it increasingly difficult for politicians to get away with lying and corrupt deal-making. That is not to say that such corruption has decreased of late. If anything, it seems to be worse than ever, but I believe it cannot possibly continue under the glaring sunlight that is the internet. I am optimistic that the ubiquitous interconnection of our society will at least result in a drastic transformation of government, if not the emergence of an entirely new form of government, that will improve equality and fairness in our society.

Some of these ideas, like the changes we will see in government, will not happen overnight. Current institutions are benefiting enormously from the way things are and so are stubbornly entrenched. The fear of change is palpable. Yet change is inevitable, and as the old guard retires, new ideas enabled by new technology will take hold. True change will only come when those that grow up with the new technology effect that change. The internet is quickly maturing so we shouldn't have long to wait. We live in exciting times.

Making the Time for Mastery

"All we have to decide is what to do with the time that is given us."

-Gandalf in The Fellowship of the Ring

"Time is the coin of your life. It is the only coin you have, and only you can determine how it will be spent. Be careful lest you let other people spend it for you."

-Carl Sandburg

How do you spend your time? Or better yet, what do you want to accomplish with your time? For a lot of people, the answers to those questions are entirely different. Some people are not doing what they want to do; some are doing what they think others want them to do; and still others, sadly, are not doing anything at all. Yet, knowing what you want to do with your time, and actively pursuing it can be one of the best ways to achieve happiness in your life. So bringing the answers to those two questions closer together will likely make you happier.

The Time To Mastery

The problem is that essentially anything worth doing that will lead to more happiness, is going to take substantial time and effort. Most people won't want to make that commitment, but it's worth it. The journey is more valuable than the destination, and for me the activities that do not have a destination are the most satisfying. Because if there is room for improvement, then there is more opportunity for enjoyment. Such a pursuit inevitably results in a lifelong love of learning that can either be focused on one interest or spread across many.

To really get a sense of fulfillment in certain pursuits, you'll want to reach a level of mastery beyond mere dabbling. You don't decide to start playing the piano to tap out chopsticks for the rest of your days. You don't decide to pick up skiing to bomb bunny hills with no style or coordination. You don't decide to learn to program so you can write for loops that count for you. No, the fulfillment in these pursuits comes from mastering the craft to the point where you can make the tools sing. Then you are no longer learning the tools, you are learning to create beauty and push the tools to their limits in new and fascinating ways.

What kind of time commitment are we talking about here? In Outliers: The Story of Success, Malcolm Gladwell makes the claim that it takes about 10,000 hours of dedicated practice to master a new skill such as learning to program or playing a musical instrument. That 10,000 hours would be equivalent to 5 years of full-time practice, 40 hours per week. Think about that. If you put in that much time to learn something, you would probably get quite good at it. Considering the few skills I feel that I've mastered, I would say that it sounds about right.

Now there are some caveats to this time estimate. First, the type of practice matters. Take learning to play the piano, as an example. If you quickly learn to play a few songs, and then only play those songs over and over again, you will not achieve any kind of mastery even if you log 10,000 mind-numbing hours of Hungarian Rhapsody on the ivory keys. Dedicated practice does involve reviewing what you're good at, but the majority of your time should be spent learning what you don't know and practicing what you're not good at. Another major focus should be constantly practicing the fundamentals until they feel effortless. Both of those aspects of practicing are hard, but necessary to keep improving, and overcoming the challenge is definitely rewarding.

The second caveat is that mastering a skill does not necessarily make you an expert, depending on how popular or expansive the field is. To be considered an expert pianist or software engineer, logging 10,000 hours will probably not be enough. The former is too competitive and the latter is too broad. On the other hand, logging 10,000 hours of speed cubing might be enough to get you near the top performers. In any case, whether you're learning piano or speed cubing, what that much practice will give you is fluency in the skill. You'll be able to do impressive things and creatively experiment, which will be immanently satisfying.

Where Does The Time Go?

Alright, 10,000 hours is a lot of time. How can you carve out that much time from your busy schedule to practice something you want to learn? Let's take a look at my own way of parsing time, see how I fit in learning a new skill, and how long it would take to master that skill. Breaking up a week of time, or 168 hours, should give a good estimate since one week is much like any other.

• I generally need 8 hours of sleep a night so that leaves 112 waking hours per week. 
• I have a full-time job with a short commute and some morning prep time that adds up to about 50 hours, so that leaves 62 hours of home time. 
• I have a young family, and while the kids are awake, my priority is with them. Between weekends and weeknights, I devote another 44 hours to my family, leaving 18 hours per week.
• This blog takes an average of 6 hours per week, and that leaves just 12 hours per week to learn something new.
• At 12 hours per week, or 600 hours per year, it will take me about 16.5 years to master a new skill!

Wow, I didn't expect it to take that long, but there are some opportunities hidden in that analysis. First, I'm lucky enough to be in a career that I love, and it also happens to require almost constant learning. That means if I'm consciously practicing to become a better programmer at work, I can become better at my job, adding value to my company at the same time that I'm personally benefiting from improving my programming skills. I still need to read outside of work to know what I need to do to practice, but if I actively apply what I learn at work, that constitutes dedicated practicing.

Also, as my kids get older, we'll be able to learn and practice new things together, with the dual benefit of family bonding time and group learning. Both aspects will give us extra motivation to learn more and have more fun doing it.

Finally, writing this blog is really part of further improving my programming skills, and attempting to improve my written communication skills. Therefore, I would count those hours toward learning skills I'm already working on. But that last number is quite the shock. If it takes 16.5 years to master a new skill, outside of career related skills, I can expect to master a grand total of 2 new skills before I retire, maybe 3 since I assume I'll have more time as I near retirement. Of course, once retired, I'll have time to master new abilities to my heart's content.

Forget the Bucket List and Make a Mastery List

It's a little disheartening to think that I'll only master a handful of skills in my lifetime. Since I'm more interested in mastering a variety of things than focusing purely on one, I have to be carefully selective of what I spend my time on. Some people have a bucket list of things they'd like to do before they die. Instead, I have a mastery list, and I think of the things on that list as belonging to three broad categories:

1. Skills mastered, and still improving: Software Engineering, Digital Design, Mathematics, Alpine Skiing
2. Skills learning, but not yet mastered: Tennis, Golf, Chess, Guitar, Juggling
3. Skills to learn, but not yet started: Piano, Physics

Overall, that's a pretty short list. Surprisingly, I couldn't think of many things I want to master but haven't started yet. I guess that's a good sign that I'm doing what I want to do with my time. One thing's for sure, though. Any one of those skills could fill a lifetime, especially the science and engineering fields. I purposefully left those as generic as possible, even though I have only mastered narrow areas of them. The joy of those fields comes from their limitless potential for learning new and challenging things. That precludes having a category for completely mastered. If I'm not improving in something, I'm stagnating and losing any mastery I may have achieved.

Now how to manage that 12 hours per week, or more if you're lucky? Well, you certainly shouldn't be watching any TV. I don't and haven't for years. I would suggest a tweak of the advice from Glengarry Glen Ross - Always Be Practicing. Make progress every week. If 12 hours a week seems like too much time to devote to one thing, try splitting it between two or more things, but if you want to make forward progress, you shouldn't let a week go by without practicing!