Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Ten years ago I had the biggest cultural shock of my career. It wasn’t witnessing poor collaboration across IT development and operations or whatever tech fiefdom you care to mention. Nope, these were minor issues compared to those facing me on a really interesting project – integrating Industrial control systems with IT management.
Working with a team of product managers, coders and sysadmins, our task was to develop a standards-based application for monitoring a physical research facility. When I say monitoring, I’m not only talking about network and systems management, but also managing equipment operating in the ‘real world’ – like manufacturing robots, pressurized compartments, control networks, smart meters, alarms and controllers.
It was a new experience for the team. Not only were we faced with a new set of technical challenges, we had to quickly become familiar with foreign sounding terms (to us anyway), like programmable logic controllers (PLC’s), supervisory control and data acquisition (SCADA) and human machine interfaces (HMI’s). Add to this a very different set of stakeholders – like industrial and plant engineers, and the initiative started off resembling something closer to a multi-language speed dating session (only without translators) than it did a traditional IT development project.
At the time projects like this were a rarity because the worlds of operational technology and traditional IT was seen as separate. Process control and electrical transmission systems ran on separate control networks with proprietary protocols, whereas IT ran on a business network and primarily handed back-office business systems – the two domains were as different as ‘chalk and cheese’.
But all that’s changing courtesy of the Internet of Things and a software-driven economy.
Today, organizations are realizing that standards, unified architectures and software applications can be used to connect and integrate the devices and sensors in the physical world with IT systems – the benefit – more seamless end-to-end business processing.
Take for example manufacturing. If plant production equipment can be better integrated with consumer demand analytics, then optimum maintenance windows can be identified. Similarly in energy and utilities, integrating the physical world of smart meters with back-end IT customer information applications is a must for efficient billing.
This convergence will drive new approaches to developing and operating applications. The DevOps movement with its focus on teamwork and collaboration to shorten delivery cycles while improving quality might just be the answer, but there are some serious challenges to overcome – not least cultural.
You might think you’ve solved the great IT divide with DevOps, but that’s nothing compared to the cultural chasm you’ll need to bridge across the world of industrial operations and IT. And no amount of IT ego will help, which I learnt the hard way when I was once given a sharp lesson in what ‘real-time’ processing actually is business by a group of chemical engineers – boy was I humbled.
Technology beat up aside I believe there are many ways DevOps can help address the challenges associated with connecting and integrating operational technologies – especially:
- Standards guidance – since operational systems work in isolation and use proprietary systems and communication protocols designed for specific control system networks (e.g. Modbus and BACnet), DevOps teams can help provide advice on new standards and integration platforms.
- Knowledge and Expertise – DevOps teams can provide expertise in the selection of new software platforms that are now available to build and run Internet of Things applications.
- Security and Risk Management – as more and more software finds its way into sensors and operational control networks become connected to the Internet, DevOps and security specialists will have a key role in identifying and mitigating many new security risks.
- Data Management and Analytics– connected sensors and control systems may result in the need to capture and analyse huge quantities of data. DevOps teams can help in areas such as application architecture and design, testing, network capacity and cloud automation.
As operational and information technologies converge, I believe the purview of DevOps will increase considerably. Now teams might include technology constituents from other areas of the business – like engineering and logistics specialists – each bringing new skills to the table that when coordinated with IT expertise in areas like mobility and cloud computing has the potential to drive huge efficiencies.
Application management will change also. Development work will increase, but so too will need to bake quality and resilience into every release. This is especially important because as processes become more integrated (from the factory floor sensor network to a mobile enabled materials management system or from healthcare diagnostic equipment to patient records applications) any minor defect, security issue or performance problem can trigger more catastrophic business failures.
When I look back to my own teams foray into managing ‘real world’ technology, I realize that while we never had DevOps to guide us we still used many principles. Success came when we built a cross-domain business team that maximized a unique set of technical strengths, but also understood each other’s weaknesses.
So DevOps teams take note – as technology convergence accelerates, the goal for many industries won’t be just bridging the IT development and operations divide and releasing more and more software, it’ll also be about developing a new range of end-to-end digitized business processes – from the factory floor to the boardroom – that’s a bigger deal and where success will ultimately be measured.
