This is the fourth installment in this series on DevSecOps. Read the first installment, on static analysis, here the second installment, on source composition analysis, here, and the third installment, on dynamic scans, here.
Dynamic testing looks at the running application, poking and prodding to see how it reacts to known vulnerabilities. A complete dynamic scan watches a test runner as it runs against the application and tries all points of entry it can find, normally a highly automated process that takes a while.
Where dynamic scans fall short is in pinpointing the problem within the application. Knowing that things went wrong when a given CVE was exercised against the app is one thing, but knowing what went wrong and where it went wrong is much more useful to make development teams productive. This is where interactive application security testing (IAST) tries to lend a hand. Using tooling in the application, interactive scans can offer the exact module/source file/shared library (etc) that caused the error.
Generally speaking, interactive tests are best run at a targeted subset of an application, finding details needed to resolve issues the organization is already aware (or suspect) exist. In our test environment, we use interactive scans to zero in on issues that were detected by static and/or dynamic scans.
It is likely that the interactive testing market will be the one to continue to move down the line to full-on security monitoring and/or it will encompass a growing list of environmental issues—it is, after all, designed for a running application to get exercised and find the issues. It is not a stretch to see these tools move in the above directions, and indeed you can see the beginnings of these moves in different vendors.
But for today, the things-you-want-to-look-for list is here. My usual disclaimer applies … Through my (and others’) work for ASG, we’ve turned up a lot of information about DevSecOps tools and what to look for. We don’t have room in these blogs to delve into the depths of that research, but I’m offering an abbreviated hit-list of things to look for when moving toward this type of product. There is a lot to look for, and please be aware that this list is just to get your journey jump-started.
- Language/framework support: It doesn’t need to be as tightly integrated as language-dependent static scans, but the tool still needs to support your specific development efforts. Support for Ruby/RoR, for example, is important if that is your dev platform of choice.
- Source of vulnerabilities: Like other security tools, understand what interactive testing tools cover and how it is covered. Currently, most of these tools count on an external test running tool to actually run the test, while the dynamic scan watches what is happening inside the application.
- Toolchain integration and support: Can the tool be called from your build/test toolset? Does it have integrations to feed results back into your build/bug-tracking systems? While these tools offer good interfaces to see what is going on, the explosion in tools that DevOps brought is making us dashboard-weary. Make certain your centralized code quality information store can be used as the single pane (or pain, depending upon implementation) of glass for initiating scans and reporting results.
- Test tool support: As mentioned above, most of these tools are watchers that rely on another source to run tests. Make certain that there is some manner of coordination between these two so the test runner can be kicked off from the dynamic scan or vice-versa. More manual steps aren’t needed here and just slow the process.
- False-positive rate: Interactive testing still struggles with false positives, though these tools are getting better. Have an understanding about average false-positive rates of the product in question, then ask about customization options to reduce this number in your given environment without reducing security posture.
- Cost to configure/maintain: As is probably obvious, tooling an application to look inside of it is a step beyond normal production needs. Make certain the overhead of adding interactive testing to a project and the cost of integrating are worth the results to be had.
Vendors call it “interactive” because it watches the interaction of the pieces of code inside the app as it’s running. Many customers call it “interactive” because the “tool” used to run apps for testing ends up being people “interacting” with the application. Both are somewhat accurate, though automation is coming apace, as it is for everything software testing.
Choose a product that has enough automation today to suit your organization’s needs, and is progressing to wow you. Of all the security tools out there right now, interactive holds the greatest promise for wowing us. Since it is happening at runtime and can see inside the application, everything from environment to installed applications on the host could be covered, and that’s where some vendors are headed. An integrated environment security test tool with deep knowledge/reporting that includes knowledge gained from static scans and source code analysis is on the horizon—and, given the direction of all of DevOps, making that solution fully automated and reporting back to build systems is coming also.
Interactive won’t replace the other steps in DevSecOps, but it will replace some of their functionality—or more often enhance some of their functionality. If a static scan notes a potential vulnerability, dynamic could verify if it is a real vulnerability before a human ever sees a report on the issue, for example.
I’ll end this as I’ve ended the others: You all are kicking rear and taking names in a tough IT environment that exploded into near-100% remote for most of you. Keep kicking rear and don’t forget security. It is more important as your organization becomes geographically dispersed because now applications that were “secure enough” for the internal network are shared publicly. Even with infrastructure protections such as VPN in place, more caution should be applied to those applications simply because they are more exposed than “must be on our LAN” allowed for.
Most users forget that in the end, someone is running those servers and keeping things working. So from me, for all those who’ve never seen the inside of IT, thank you. And keep rocking it.
