General Purpose Fuzzing

No matter how bad the quality of mutation-based fuzzing is, there are practical use cases for it. Look at all the hacker tools available and you will notice that most fuzzers just grab files (jpeg, mpeg, doc...), traffic captures (pcap, pdml, ...) or XML-like schemas (schema, dfd, ...) and generate tests from these "specifications".

So why are traffic capture fuzzers so powerful? The reason is speed! I selected a wireless access point and listened to it using a network analyzer while it was booted up. During the boot process, I caught a number of packet traces, which I could then directly export to our brand new traffic capture fuzzer (or in the case of XML protocols, into our XML fuzzer), and just after 5 minutes, I had all test running, and less than 100 test cases later, I had the wireless router booting for the first time.

General purpose fuzzers expedite the process of creating fuzzers as they automate the entire test generation process. Test coverage is minimal compared to full model-based fuzzers, yet it more than suffices to find a few bugs in modern communication products. Highly recommended!

The Fuzzer That Does Not Fuzz

From: http://www.codenomicon.com/news/newsletter/archive/2009-08.shtml

"The fuzzer that does not fuzz", was how Codenomicon test tools were described at Black Hat USA 2009. Without necessarily knowing it, the speaker made the biggest compliment to our tools anyone has given for years, if ever.

Before 1998, all fuzzers that at least I know about were entirely stateless, and purely random. You should not really even consider this approach any more.

After 1998, in the PROTOS project, we described an approach where no randomness was involved at all. We called it Robustness Testing, based on definitions we heard e.g. ETSI use for such a testing approach. Other names for similar approach are grammar testing (used e.g. by Wurldtech) and syntax testing (used by testing specialists everywhere).

In PROTOS we noticed that if a protocol was modelled using dynamic and thorough state machines and message descriptions, there was no need for randomness anymore. Actually, the incremental benefits of adding random tests to the systematically built tests was so insignificant that eventually we just left them out entirely. Everything was carefully optimized. Test execution times were extremely fast (from minutes to few hours), and test coverage was much better than with other techniques, even those in use today.

After almost ten years, block-based fuzzers were invented. They are a kind of cross-breed between the purely random, non-protocol-aware fuzzers of the early 90s, and robustness testing tools that are entirely based on protocol models and systematic test generation. A block-based fuzzer adds enough protocol awareness to its minimalistic model and state diagrams, to be able to somewhat limit the amount of random or semi-random changes it does. Why did the inventors include any randomness at all? Because a fuzzer is supposed to do random testing - or perhaps that is just what people thought.

So when someone calls us a "fuzzer that does not fuzz", they are finally understanding the difference between a fuzzer, and a Robustness Testing tool. Even though finally in 2008 we decided to call our tools fuzzers also, they really isn't anything fuzzy about our tools. And we are proud of that!

XML Fuzzing

Information regarding the XML flaws in the Codenomicon pages here: http://www.codenomicon.com/labs/xml/

Automated Whitebox Fuzzing, the Microsoft Way...

... here.

Finally, a market study on fuzzing

Everyone has heard rumors of this and that company using fuzzing. Finally the real data is available: Codenomicon has invited Forrester and Cigital to share their findings.

Check out the webinar http://www.codenomicon.com/resources/webcasts/20090331.shtml

Fuzzing In The Media

So what do people think about fuzzing? If you are a security specialist, you might think that fuzzing is a cool tool in the hacker toolbox. Or if you are in quality assurance, you might think it is just another protocol modelling tool, but with a specific purpose for finding security tools. But for the rest? The answer is simple, they have no idea what fuzzing is.

I was speaking at a press conference for Infosec London, and my personal experience there supported my past thinking. Only five out of twenty-plus journalists had ever heard about fuzzing. And these people write about security topics in their publications! It is a long road to change this, and we need everyone's help in that. That is actually the only reason why I wrote a book on fuzzing, and why I have one too many blogs on the topic. Please join me in evangelizing on fuzzing, to change the world for better!