Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Modern operations teams today are at the cutting edge of automation. DevOps and SRE practices are entirely based on an automation-first philosophy. CI/CD is fully automated and the next generation of automated testing is now being adopted.
However, these same teams are still running their human processes based on hastily thrown-together wiki pages. The increase in remote work over the last year has amplified the problem. Engineering teams that were used to quickly getting the right people physically into a war room now struggle to find the people with the right knowledge to troubleshoot or resolve a problem.
Teams with ample resources have built tools and bots for DevOps process automation and orchestration. These tools can help automate things like getting communications channels set up quickly, with the right people, the moment an incident is identified. Or, they may automate actions across the DevOps toolchain that would otherwise live as static documentation for an on-call engineer to try to decipher and execute. But only a handful of organizations have the resources to build this kind of automation technology in-house.
These higher-level DevOps processes are ripe for innovation similar to robotic process automation (RPA). Companies like UiPath and Automation Anywhere have shown that there’s an enormous market for this in every business unit and industry. Process automation for DevOps needs to be another layer of automation available to any engineering team. Automated remediation of incidents and resolution of service desk tickets has to be the future of this space. However, there are important differences in how automation is optimized for DevOps versus the application of RPA for business functions.
Where RPA Fits on the Automation Spectrum
RPA is great for solving deterministic problems – those that can be resolved in the same way every time you run into them – which are fully automatable. An example of this type of end-to-end automation is any back-office task, which are consistently good candidates for the application of automation. It’s the reason why fields like HR and finance have made some of the greatest progress in automating business processes to date. Every time you close the books, for example, the steps are pretty similar.
Process automation for DevOps and SRE, however, is on the other end of the spectrum. Although teams may benefit from RPA for simplistic, repeatable tasks, many DevOps processes are dynamic and unpredictable, making full end-to-end automation too often fallible. The complexity of modern technology stacks and infrastructure requires processes to be agile and adaptable to change, rather than deterministic. This is where it’s important to enable humans to step into the spectrum of automation. Human-in-the-loop automation – the idea that lets humans be in the cockpit while letting machines provide guidance and support – is the future of DevOps automation.
Applying Process Automation to DevOps
To use automation in a meaningful way, DevOps teams need a mix of what RPA provides with the added element of human judgment. Human-in-the-loop automation can be applied to numerous DevOps processes including incident management, service requests and continuous deployment and delivery.
Rather than automating from end-to-end right from the start, DevOps teams should use incremental automation to strengthen and build trust in automation. This starts by codifying free-form processes so teams can more easily identify predictable steps that can be automated. Then teams can mature to more complex workflows, automating pieces within processes and incorporating human checkpoints for areas that will need judgment or approvals.
As teams incrementally automate, they build up sustainability – in their systems and their roles – in high-stress environments and are able to orchestrate processes across the continuum. The goal is to find the right balance between leveraging machines to make lives easier while creating room for human judgment and adaptability.
Level-Up DevOps Automation
Looking ahead to the next few years, what the RPA movement did for back office tasks will also happen for DevOps and SRE. Technology leaders can take notes from the RPA movement and apply them to eliminating the toil of modern production operations. The issue is that, in DevOps, many problems are unique.
The next evolution of DevOps automation will force us to look at how we can model the real world in a more holistic way, which includes incorporating human judgement and manual execution of non-repeatable tasks while automating repetitive steps.
