You’re Thinking Software Wrong

Like any industry, DevOps fosters its own unique culture. And like any culture it resists change, it has inertia. “This is how it has always been done” is an incredible force to reckon with.

Even though the realities of DevOps have changed dramatically in recent years with the advent of new tools and services, many ideas that have been ingrained in the culture will probably still be with us for ages. Some of these ideas are not only wrong, they border on being dangerous. One such fallacy is the sacred rule cherished by development teams and system administrators: “NEVER deploy on a Friday afternoon.” That is, unless you want to spend your entire weekend fixing bugs. A more pernicious example is the division of labor between us, humans with “root access” to machines, and the robots, the tools teams create to make their processes less manual. These days you’ll still find many development groups that delegate the menial tasks to automation, but keep the most dangerous, infrequent tasks to human operators. These are two prime examples of “thinking software wrong.”

These ideas are not only incorrect, they can be crippling to an organization, causing delays in software development and leaving the company vulnerable to data breaches, for no good reason. These fallacies are what we refer to as “broken culture”—what happens when culture is deeply misaligned with the interests of the organization, the individuals who compose it and its stakeholders. It usually manifests when historic cultural norms prevent members from embracing new and better ways of doing things.

Fear of Friday deployments is our first example of broken culture. This idea has taken a strong hold in DevOps culture because developers haven’t trusted their own software and systems enough, and they don’t want to spend their Friday nights (or the whole weekend) debugging a production problem. Fair enough, but there is a better way.

Let’s look at another example: the Equifax breach. This incident—which cost the company billions of dollars and continues to haunt consumers to this day—happened because a certain procedure mandated that a specific piece of software must always be updated by a human. Since humans are prone to error, this lone simple procedure (designed and supported by years of a broken DevOps culture) resulted in a devastating data breach. Once again, there is a better way.

Deploying is not the same activity it was a decade ago. Advanced automation tools are drastically streamlining deployments, significantly reducing the chance of error and speeding the time it takes to identify and correct any errors that do occur. This means companies can deploy any day of the week and any time of the day, worry free—and can let the deployment date be dictated by what is best for the company and its customers, not the weekend plans of developers.

The same concept applies when it comes to updating software. A better way is to leverage the automation tools that are now readily available, and allow organizations to have robust management of compliance requirements without human error or delay.

In addition to embracing better tools, here are a few other ways companies and developers can rethink software for the better:

• Software must be valued as more than just the cost it takes to create it, or to replace it. Code should be an asset, not a liability.
• Software should be thought of as constantly changing—a dynamic system with which people, machines and processes interact continuously. When this never-ending process is supported by the right tools, deployments are easier and managing and updating software is seamless and secure.
• Code by itself is useless. It needs infrastructure to build it, and data to create something meaningful. Therefore, if you only test the code, but are not managing your infrastructure in the same way, you’re not really testing the software system.
• Companies can dictate the cadence at which they create new software, but they do not have this same luxury when it comes to fixing broken software. Identifying and fixing problems must be done as soon as possible.
• Every new update or fix must be version controlled so that everything is auditable. That includes changes to infrastructure configurations!

Once companies truly understand these fundamental truths about software and embrace the modern tools available to them, they can start to shed the aspects of DevOps culture that are limiting or damaging.

Companies can deploy new apps and features on Fridays, confidently and without issue. Companies can even rebuild an entire infrastructure from scratch, in minutes, while applying all security updates for every single service and library—and be completely sure nothing will break. If you can’t do that—or don’t believe you can—you’re thinking software wrong.

— Ori Pekelman