One of the key mantras of DevOps is “continuous.” Effective DevOps relies on automating routine tasks. Any step in the process that requires human intervention or manual input becomes a roadblock that can slow or halt productivity. To the extent that it’s possible, anything and everything should be automated—and that includes testing.
Test is like the Rodney Dangerfield of DevOps—it gets no respect. It should, though. Test—and test automation—is a crucial element of a successful DevOps program. A recent Gartner survey found that 50 percent of respondents listed test automation as one of the top enablers of DevOps success.
Joan Wrabetz, CTO of Quali, is focused on automating test capabilities and ensuring that testing gets the respect it deserves in DevOps. DevOps isn’t really a process with a beginning and end. It’s a continuum—a self-perpetuating cycle. When test teams take the lead, they need to find ways to push automation back into development and forward into staging and production. “In fact, the problem of test automation is particularly difficult, and this means that the team that implements test automation is in a great position to lead DevOps initiatives beyond test,” Wrabetz says.
That self-perpetuating cycle “starts” with planning and development and “ends” with deployment into operations, but that flows right into monitoring and management, which leads to discovering issues and identifying new feature requests—and that takes you right back to planning and development. If done properly, test falls somewhere in the middle of that cycle, so it’s imperative to streamline and automate that functionality as well.
“Sometimes test drives DevOps. Test organizations have been focused on automation already for 10 years, so often they are the driver for DevOps,” explains Wrabetz. “When test is the focus of DevOps, it is called continuous integration. However, it is also important to work to continuously expand automation outside of test.”
Key Drivers
According to Quali, there are five key components that are essential to successful automation in a DevOps environment:
- The infrastructure. It starts with one very important characteristic that is critical to testing and often is not automated—the ability to automatically mimic the production infrastructure in which the software (or hardware) being tested is expected to run. Having the ability to give each tester (or set of automated tests) their own personal replica of the target production infrastructure—a.k.a. a “sandbox”—is critical to ensuring that the software coming out of test can be deployed automatically and continuously into production. Automating tests is not nearly as difficult as automating the underlying infrastructure in which those tests run.
The production environment workload. Often not considered in automating testing is the requirement to mimic the workloads that would be common in production. This includes the network traffic, other application workloads and network security profiles. These activities need to be automated within the sandbox to create a more realistic environment for testing.
The Tests themselves. Of course, the tests themselves also need to be automated and there are a variety of tools to assist in this process. As software gets more complex, this continues to be a challenge, with mobile testing replacing GUI testing as the most challenging type of testing to automate.
Reporting and result analysis. To provide continuous automation from development through testing to delivery, it is important to see and analyze test results automatically.
Tool Integration. Automation into and out of testing needs to be enabled. This means sandboxes and test automation suites need to be API-driven so that they can be started by a previous tool in the DevOps tool chain and can also initiate a tool that follows testing in the tool chain.
Show Some Respect
Testing is a crucial part of the DevOps continuum. It needs to be done properly and effectively, but it also needs to be a streamlined process that drives DevOps forward rather than a hurdle that slows everything down.
