Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
A while back, I overheard someone asking why many of the ‘cloudy people’ have moved on to DevOps.
One reason is certainly that cloud is at the heart of DevOps. Many of the grounding principles of DevOps – rapid iteration, agile development, automated testing, continuous delivery and continuous integration – are barely even possible without cloud. However, I think it runs even deeper than that. DevOps and Cloud are both fundamentally enabled by the same technology I have been working with my entire career – automation.
When IT leaders get excited about cloud, it is mainly because of automation. Sure, cloud providers get excited about essential characteristics like broad network access and resource pooling, because that is how they maximize reach and minimize costs. Similarly, CFOs get excited about the measured service, because that means they only pay for what they use.
However, according to a recent Luth Research study, most IT leaders prioritize the higher-order values enabled by on-demand self-service and rapid elasticity – faster innovation, superior performance, better scalability and higher resiliency. These capabilities are deeply rooted in automation – of provisioning, deployment, configuration, scale-up/scale-out and other processes.
Similarly, DevOps – beyond a certain point – also builds on a foundation of automation. As Rackspace CTO John Engates noted in his recent column in this journal, “DevOps is more than a set of automation tools.” He is right, of course. You can achieve a great deal through cultural and structural changes – better collaboration, shared goals, team building, process improvements, etc.
However, tools for release automation, configuration automation, process automation, test automation, service virtualization and provisioning – just to name a few – are adding tremendous value with a rapidly maturing DevOps automation toolchain. Indeed, in recent research from Vanson-Bourne, over half of senior IT decision-makers worldwide rate automation as the most important DevOps component – above such putative stalwarts as agile development, collaborative teaming and process alignment.
Looking at the history of enterprise automation, and considering my own experience, I believe there are many common principles that you can apply to both cloud and DevOps. Here are just five of my top tips:
Don’t Automate Bad Processes
Automating bad process just makes bad things happen faster. This seems obvious, but I often see automation that only encodes (and accelerates) a bad existing practice. To avoid a high-speed train wreck, first identify, catalog, document, review and refine key manual processes to make sure you are automating the right activities, at the right time, in the right way. Solve for simplicity, speed and throughput, even before you apply automation. Remove friction across silo boundaries – departmental, technological and disciplinary. Start simple, be inquisitive, and iterate methodically. Then make sure you automate deeply, thoroughly and efficiently.
Cross-skill Existing Resources
Great people are hard to find and salaries for cloud- and devops-relevant skills are amongst the highest in IT. This explains why, according to the same Vanson Bourne research, over 70 percent of DevOps leaders are retraining existing personnel, while only 50 percent are hiring new resources. Look for staff with a history of automation in their resume, through multiple waves of technical change, rather than specialists in one automation technology or product. Find the IT equivalent of a “good athlete” rather than a “position player”.
Security Cannot Be an Afterthought
Automation systems and the staff responsible for them have always needed highly privileged (and frequently shared) user IDs, whether automating a mainframe IPL or scripting a production code release. This is an audit red flag, and can result in high-impact incidents, whether accidental or malicious. Identity and access management provides essential audit and control, while giving users (including automation tools and APIs) easy access to the resources they need (locally and in the cloud), when they need them.
Document, Measure, and Backup
If you do it right, automation quickly becomes pervasive and mission-critical. Without documentation, you create inscrutable systems, tribal knowledge, the hero and other ‘anti-patterns‘ in your Cloud or DevOps machine. Without measurement, you cannot know if (or when) complex autonomous systems run amok or stop working. Without backups, critical systems are prone to irrecoverable failure. Cloud and DevOps teams must value and prove traditional disciplines like documentation, measurement and backup – painful as they may be.
Build Awareness into Automation
As automation becomes mission-critical, it must be resilient to errors, failures, exceptions and other unexpected conditions, and still deliver a correct result. Remember when ‘autonomic computing’ was a thing, when we dreamt of ‘self-aware’ systems? Similarly, concepts like ‘desired state’ and ‘idempotence’ go back at least as far as mainframe and server automation. Similar concepts apply to cloud computing and DevOps too. In both, it is critical that automation routines are aware – of their own state and of the surrounding environment – so that they can test for and accommodate unexpected conditions, and still deliver the expected results.
Given the depth of history in automation, and the many parallels with modern computing, some may claim that cloud is nothing new, that we have been doing that since the mainframe; while others will say something similar about DevOps. I think they are wrong, but there are definitely parallels, and George Santayana has a point – we should be at least informed by history, if we are going to try to write the future, while avoiding the mistakes of the past.
