Developer’s Guide to Fuzz Testing

Fuzz testing continues to grow in popularity among developers within the open source community. Google’s open source security (OSS) team recently reported finding more than 40,000 bugs in 650 open source projects through the use of fuzz testing. But if you have no clue what fuzz testing is (yet), don’t worry. You’ve come to the right place.

What is Fuzz Testing?

Fuzz testing is a dynamic testing method for finding bugs and security issues in software. During a fuzz test, a program or a function under test gets executed with invalid, unexpected or random inputs to uncover unlikely or unexpected edge cases. 

Sponsorships Available

Value of Fuzzing for DevSecOps 

As we know, DevSecOps extends the approaches of DevOps and Agile by adding suitable security testing methods alongside every phase of the software development life cycle (SDLC), creating a dynamic and continuous testing process. 

Part of that testing process can and should involve fuzzing to detect security and stability issues at all phases of the SDLC and empower developers to ship software quickly and securely. For example, to prevent regressions from going live by running automated fuzz tests on every code change.

Especially in the automotive industry and complex embedded projects, where a lot of untrusted and complicated data gets processed, fuzzing brings excellent benefits to DevSecOps. Indeed, developers can apply fuzzing similar to unit testing to automatically test their security-critical modules for potential vulnerabilities. But with fuzzing on an integration level, they can also cover all the complicated and dangerous edge cases that can occur in the interaction between different modules. 

Fuzz Testing Benefits

Fuzzing code generates thousands of automated test cases in a second, with the fuzzer receiving in-depth feedback on code coverage and the program states reached to simulate the interaction of different modules during the execution of the source code. The fuzzer can adapt and mute its inputs based on runtime feedback in the following iterations to maximize code coverage. Some of the benefits are:

1. (Almost) no false positives: If you find a crash, fuzzing provides you with the error source and the malicious input that caused the issue. So, you can be sure that each finding is a real vulnerability.
2. Debug reproducible findings: Modern fuzz testing tools will provide you with the input causing the issue, including the source of the error. With this, you already have a lot of helpful information to reproduce the bug and load it directly into the debugger. 
3. Find issues in forgotten edge cases: Because fuzzing is executing a program with invalid or random input, it also uncovers unlikely and unexpected edge case errors that would not otherwise have been revealed.

Who Uses Fuzz Testing?

As cybersecurity regulations and standards continue to expand globally, more and more organizations are extending cybersecurity programs and running automated security testing before they ship software. Whole industries, especially those that involve advanced quality and security regulations, are setting standards for security testing that recommend fuzz testing, including automotive, aviation, finance, healthcare, telco and energy.

Digging deeper into the automotive industry, for example, a variety of International Organization for Standardization (ISO) and other organizational standards now recommend fuzzing:

• ISO/SAE DIS 21434: Road Vehicles – Cybersecurity Engineering
• NIST Security Guideline for Software Verification (Executive Order 14028)
• UNECE WP.29: United Nations World Forum for Harmonization of Vehicle Regulations
• ISO 26262: Road Vehicles – Functional Safety

Best Practices

The most effective way to do security testing, fuzzing included, is to do so continuously. This means integrating fuzzing right into your CI/CD pipeline. Doing so creates quick feedback cycles so developers can fix security vulnerabilities before the code is shipped. Integrating fuzzing tools into code hosting systems like GitHub is also valuable. Doing so allows communication and alerts to be shared across your DevOps team as soon as a problem is uncovered, which helps to facilitate fixing vulnerabilities. 

What Bugs Can I Find?

While the practice remains relatively new, fuzzing has already discovered thousands of bugs across different use cases. Bug and vulnerability types discovered include:

• Remote code executions
• Injections, which can be triggered by untrusted inputs 
• Memory leaks: Incorrect memory allocation 
• Exposure of sensitive data: Accidental exposure of personal data
• Functional bugs: Inputs that are not being responded to 

and many more bugs, such as undefined behavior, uncaught exceptions and buffer overflow. Learn more.

Getting Started

You can try out an open source fuzzer, like Jazzer (for Java testing) or the CI Fuzz CLI (for C/C++ testing). When you’re comfortable with fuzzing and want to use this approach in more complex environments, enterprise solutions offer added features like API fuzzing, reporting and CI/CD integration. 

So give it a shot and join the community of open source developers using fuzzing to quickly and securely ship their software.