Search This Blog

The Software Industry Grew Up While We Weren't Looking

This week I'll continue expanding on things I brought up in my first post by taking a deeper look at how the software industry has matured over the last decade or so. I noted that the software industry has come of age, in a way. When I say that, I am actually thinking about it a few different ways.

In some of the software blogs I've read, especially posts around the 2003-2005 time-frame, I got the distinct impression that many software developers believed that the practice of software design and development was immature. Developers didn't want to be called engineers because they didn't see software development as anything like other forms of engineering, they didn't think developers were professional in the same way as other engineers, or they didn't think the tools, processes, and best practices were nearly as developed as in other engineering fields.

This is by no means a complete list. I've seen countless other reasons for this apologetic or dismissive attitude, but let's focus on these three views and not overwhelm ourselves. I would say these views have either diminished in prevalence or improved significantly, and in doing so, the software industry has been growing up. Let's take each of the in turn, shall we?

So is software engineering like other forms of engineering? Of course it's not. Is nuclear engineering like mechanical engineering? Not really. Is aerospace engineering like chemical engineering? Um... nope. Is electrical engineering like civil engineering? You get the idea. These are all different engineering disciplines because they are different. They do have one thing in common, though - the definition of engineering:
Engineering is the application of scientificeconomic, social, and practical knowledge, in order to design, build, and maintain structures, machines, devices, systems, materials and processes.
In the case of software engineering, the pertinent parts of the definition would mostly be social and practical knowledge applied to systems and processes, but it definitely applies.

It seems that what had software developers rejecting the engineering label was partly a feeling of awe in creating their own rules. How could software design be engineering when you can just define the rules to be a certain way and then design to those rules. If you don't like the rules, or they don't work, you can change them. Other fields of engineering are limited by physics. How unfortunate, and not at all like designing software, went the line of thought. But software is limited by physics - specifically time and space in the form of execution speed and memory or hard disk space. Those are hard (although improving) constraints that affect software scalability all the time.

Beyond the feeling that physics did not apply, there was a feeling of building something completely new and different with each piece of software that made it seem like domain knowledge gained on one project was not transferable to the next project. That characteristic of software design seemed to set it apart from engineering. So let's take building bridges as a counter-example. If a civil engineer is always building bridges over roads, the skills she has developed are probably pretty applicable from one project to the next. If she goes from bridges over roads to bridges over rivers or canyons or even freeway interchanges, the constraints change dramatically, and so does her ability to apply what she's already learned. What happens if she moves on to designing hydroelectric dams? That's a completely different kind of design problem. Will she need to learn new things? Of course! It's not so different from moving to a new domain in software development.

Why did software developers think their profession was somehow different at a fundamental level? I'm not sure. It might have had something to do with the perspective of "What you do is easy, what I do is hard." I think the software industry has outgrown that way of thinking. Developers have realized how much software design is like engineering and accepted the engineering label because that is, in fact, what they do. They've moved on to focus on solving the pressing problems before them.

That brings us to the second reason software developers didn't want to be called engineers. They didn't feel like professional engineers. With regards to the title of Professional Engineer, most engineers are not. Let's get that out of the way right away. That title mostly applies to civil engineers and architects. Other engineers can take the test and get the certificate to make it official, but it's not required in general.

It almost seems like software developers didn't want to be called engineers because they wanted to avoid a perceived sense of responsibility or because of a fear that becoming engineers would take the fun out of software development. The rise of large software companies like Google, Amazon, and Facebook and the successes of the Open Source movement have gone a long way toward changing that perspective. Now you can design great software professionally while still having fun doing it, and it's all quite respectable. On top of that, these companies commonly call their software positions "software development engineer," or something similar. They've promoted the label, and it's become widely accepted in the industry.

That still leaves us with the issue of software engineering tools, processes, and best practices. The improvements gained in this area over the past decade have been extraordinary. We now have well defined processes that range from the more structured and controlled Rational Unified Process to the more flexible Agile Methods. Depending on project and customer requirements, the methodology can be made to order from a wide array of standard options. On top of that, best practices are available at your fingertips either in the form of books too numerous to list, or excellent websites such as stackoverflow.com.

And then there are the tools. The tools are where the maturation of software development really shines. Integrated development environments and debuggers have come incredibly far in a relatively short time frame, and we now have a plethora of options to choose from. Bug tracking and version control software has transformed the way teams plan and build software, making large projects so much more tractable and manageable than they were before, while supercharging developer productivity. Now distributed version control promises to drastically improve software code base management yet again.

These tools show the maturity of the software industry not only in their application to software engineering, but also in the fact that other engineering fields are adopting them to improve their development. While I was an electrical engineer at my previous company, we started using Bugzilla and Subversion on our projects, resulting in significant improvements in project management and quality. It turns out that bug tracking and version control have general applicability to basically any business or field of research and development. Tracking issues and change management are fundamental to achieving quality, and these tools that automate and streamline those processes were developed first for software engineering. Now they are spreading into all kinds of applications from medicine to supply chain management to customer service, and making substantial improvements wherever they are applied.

While thinking about other engineering fields accepting development methods from the software industry, it occurred to me that this was a defining characteristic of a maturing industry. The software industry was no longer just taking from and building on the other fields of engineering. Now other fields of engineering were building on the innovations coming out of software engineering. Those decade-old views on software development not being a form of engineering mostly disappeared because the software industry had grown up without us (well, me at least) noticing. Even though the software industry has matured in the last decade, it shows no signs of stagnating. New trails will continue to be blazed. Novel discoveries will continue to be made. The innovation we have seen has only just begun.