Early vs. Late-Stage DevOps Testing: The Benefits of Both

Companies that are working toward optimizing DevOps processes have over the past few years adopted a practice of testing throughout the software application design, development and production phases—a practice of continuous testing throughout the product life cycle. This includes shifting testing operations “left,” i.e. beginning testing earlier in the life cycle, to catch any software malfunctions or errors as early in development as possible.

While shifting testing left is critical when it comes to reducing the cost of late-stage software corrections, DevOps teams must also remember that software malfunctions may arise at any stage in the development process, as well as post-production.

The Early Bird Gets the (Software) Bug

Continuously testing software throughout the product life cycle is a strategy used to optimize DevOps processes. Continuously testing software from design to production throughout its life cycle provides DevOps teams with an uninterrupted flow of software testing data. DevOps teams can use this data to inform software design and development based on the performance successes and failures of previous and existing products.

Shifting testing left—earlier in the software development cycle—enables companies to catch as many software errors as early as possible to avoid potentially costly redesigns in later-stage development. Shifting left was a crucial area of focus for many of the companies featured in the recent “Forrester Wave for Continuous Functional Test Automation” (CFTA).

Shifting testing left significantly not only reduces the overall cost of software development but also hastens time to market. DevOps teams can avoid a back-and-forth pattern, or context switching, which involves switching between developing various features of the software as they are tested and verified. Context switching to test various software functions also risks fatiguing and frustrating software developers, as they may be forced to repeatedly revisit and redesign certain software features when they could be focusing on more critical aspects of the product. When DevOps teams are burdened with tight deadlines, they may end up cutting corners, creating more risk for errors that may present themselves to the product’s end user.

Late-Stage Testing: Removing the Rest of the Bugs

However, just because a software application works in its current testing environment doesn’t mean it is unbreakable. How many times have you seen a product such as a dishwasher or a fridge operate perfectly in the store during a demonstration, only to get it home and discover it doesn’t fit or the power cable doesn’t reach the same power outlet as the old machine? How many times have you ordered a product, even one that you’ve used countless times before, and opened the packaging to discover it was broken or had some kind of defect? In the same vein, how many times have you ordered an electronic device that required a charger or electronic port type that you or your computer did not possess? It’s not exactly the great end user experience product design teams had in mind when they were designing the product. For DevOps teams that neglect shift-right testing, that’s exactly the negative end user experience they are risking with their software products.

Shifting DevOps testing right, also referred to as “testing in-production,” is the opposite of shifting left. Shifting right involves testing software later in the product life cycle in late-stage development and even as far as production. This strategy enables DevOps teams to identify and fix bugs that may only arise after shift-left corrections have been made. In other words, if a DevOps team uses shift-left testing to catch bugs earlier, they will then go in and correct those errors. But how do they know that more bugs won’t arise in the production environment? That’s where shifting right comes in. In this respect, shifting left and shifting right are critical aspects of a continuous testing strategy that should be used together to perfect software validation processes.

Shift-right testing also helps to enable a seamless end user experience. A company’s test environment and end user environment may not be exactly the same. When applications are deployed to different cloud or on-premises environments than those in which they were tested, it’s nearly impossible to predict how those varying environments will affect the performance of the product.

Even if the end user environment and testing environments are the same, it’s an unfortunate but inevitable fact of life that things break. Unless software companies are truly continuously testing their products, which includes production phases, they will fail to identify and fix any bugs before their customers can point them out. While customers appreciate rapid fixes when they point out an issue, it’s a poor substitute for a product that customers never have any problems with. Companies and DevOps teams that hope to perfect software development processes are embracing a continuous testing strategy, also known as DevTestOps, and they should be careful not to turn a blind eye to the post release part of the software development life cycle by ignoring the importance of shifting testing right. While shifting left and shifting right are separate  strategies, they should not be viewed as mutually exclusive and DevOps and testing teams must work together to develop the best products possible while simultaneously eliminating the need for back-and-forth development and testing patterns.

If companies truly hope to fully optimize DevOps processes, they must embrace a practice of truly continuous testing that spans the entirety of the product life cycle from design through production. Ultimately, this not only enables developers to close the loop on their development cycle to develop more robust, effective applications, but it also provides the competitive advantage of a faster-time-to-market.