Get on-the-go access to the latest insights featured on our Trustworthy Computing blogs.
Adam Shostack here. One of the really exciting things about being in the Microsoft Security Engineering Center is all of the amazing collaborators we have around the company. People are always working to make security engineering easier and more effective. When we talk about security testing, we often focus on what it can’t do. “You can’t test security in,” and “test will never find everything.” But much like there’s code that’s easy to get wrong, there’s code that’s hard to test. Writing code to be testable has a long history, and one we don’t often talk about in security. Today’s post is from Hassan Sultan, who’s responsible for one of our internal fuzzing tools. We hope it inspires you to think about the question “How can I make the security of my code more easily tested?”
And here’s Hassan:
Security testing is an integral part of the software development lifecycle. At Microsoft, the biggest part of the security testing done is usually implemented through a technique called fuzz testing: sending unexpected input to the product and checking whether it behaves in an acceptable way (i.e. it doesn’t crash, hang, leak memory…). We also use other techniques such as static source code analysis but today we’re going to focus on fuzz testing and how you can best make use of it.
Almost every software company and every software project has to perform within constraints, they can be financial, the project has to be completed within a set budget, or time-driven, the project has to ship within a specific timeframe. The corollary is that the product must be of the highest quality possible within those constraints. How then can you perform efficient, quick and cheap security testing?
One approach we have started using at Microsoft is to change our engineering and test engineering practices to make fuzz testing easier, it’s a little bit of additional upfront work but with great savings in terms of time and resources quickly appearing over the life of the project.
There are two popular approaches to fuzz testing, considering data exchanges between a producer (the software sending data) and the consumer (the target software processing the data):
A couple of things are obvious when comparing these two approaches:
The approach I’m going to talk about is based on MITM fuzzing; the goal is to develop functionality tests in a way that makes them easily reusable as producers for MITM fuzz testing, as well as to provide a bit of functionality in the actual product to make fuzz testing more efficient. This approach makes security testing much cheaper to implement, is quite efficient and allows improving the fuzzing over time without having to rewrite numerous security tests.
MITM fuzzing using functionality tests has the following drawbacks:
The approach here is thus to fix all these problems at the source, we have listed the steps required along with each step’s priority, obviously the more you do, the better, but if in a crunch, start from the top of the list and go down as far as you can.
(A test hook is a configuration option that modifies the product’s behavior when set, it can be removed before the product ships if needed)
These modifications to the way tests and products are engineered are minor and cheap to implement when planned early on and will produce tremendous benefits by:
Ultimately, using both Generation fuzzing and MITM fuzzing would be ideal, as generation fuzzing provides a few benefits that won’t be attained by MITM fuzzing(the ability to create very specific scenarios for example), but when dealing with time and resource constraints, the MITM fuzzing approach allows for efficient fuzzing that can be improved over time at a minimal cost.