Thinking About Test Cases
A lot of attention has been paid to identifying attackable surface areas, but less to the development of attack pattern libraries. When we dynamically test web applications for security vulnerabilities, how good are the test cases we’re using?Commercial web scanning tool vendors put significant research effort into this problem, but the product of this research is considered intellectual property and locked up inside the application. As users, in order to learn what kinds of test cases are being generated we would need to painstakingly record and analyze its traffic. At the time I initially released FuzzDB, most open source web fault injection tools had sets of test cases which were woefully incomplete and inadequate. There are too many permutations of symbols and encodings used in web protocols for anyone to reliably and repeatably recall all of them. As for the commercial tools, how complete are their sets of test cases, anyway? It’s not always easy to tell. What were they actually testing for? These tools aren’t just test case lists, they’re lists wrapped in complex sets of rules that determine which test cases to use when and where. After considering these details, I had some doubts about the effectiveness of the typical application testing process.
Introducing FuzzDB | Mozilla Security Blog