Skip to content

Cyber Security Benchmark Highlights Legacy Product Failures

By Contrast Marketing

September 22, 2015

Cyber Security Benchmark Highlights Legacy Product Failures

US Department of Homeland Security-funded project documents weak performance of application security solutions  

Palo Alto, Calif. – September 23, 2015  - In August 2015, with funding support from the US DHS, the Open Web Application Security Project (OWASP) published an open source Benchmark Project on application security accuracy. The Benchmark Project allows organizations to measure the effectiveness of application security solutions by providing an application with over 21,000 test cases across 11 different attack categories. It also uses code that looks vulnerable, but isn’t, to check for false alarms.

The new Benchmark Project exposes the failings of the Static Source Code (or SAST) and Dynamic Web Scanning (or DAST) product categories. The best performing products in those groups scored a discouragingly poor 33% accuracy on the Benchmark, demonstrating that companies relying on them are left vulnerable to hackers. That’s alarming given the importance of application security, and business dependence on those products. 


Security professionals have long believed that one of the main challenges with securing applications was the application security products available to them, but until now had no way to conclusively prove their effectiveness.

“By understanding what a tool can and cannot do, the OWASP Benchmark Project has the potential to positively stimulate improvements in software security assurance tools,” said Kevin Greene, Program Manager, Cyber Security Division, United States Department of Homeland Security.

High Failure Rates of Existing AppSec Products, with One Significant Exception

Applying the benchmark application consistently across application security products produced astonishing results on the accuracy front:

  • 18% accurate – Most accurate open source Dynamic Application Security Testing (DAST) product
  • 17% accurate – Most accurate commercial DAST product
  • 33% accurate – Most accurate open source Static Application Security Testing (SAST) product
  • 33% accurate – Most accurate commercial SAST product
  • 92% accurate – Contrast Enterprise, an Interactive Application Security Testing (IAST) product



Figure 1: Benchmark Accuracy Results (September 17, 2015)
Accuracy scores for products across all 11 Benchmark Project vulnerability categories.

Reevaluate Application Security Products and Programs

Using the benchmark, organizations should evaluate the strengths and weaknesses of their current application security solutions, and reconsider their options. Contrast Enterprise, which the OWASP Benchmark demonstrated is both fast and accurate, is a natural choice to augment or replace existing SAST and DAST solutions. Ask your application security vendor for their benchmark results, and contact Contrast Security ( to learn more about Contrast Enterprise. false-sense-of-application-security

About Contrast Security

Contrast Security is the world’s only application security software that quickly and accurately stops hackers from stealing data via web applications – the most successful attack vector. Industry research shows that application security flaws are the leading source of successful data breaches yet more than 90% of applications are not secure. Unlike legacy security products that do not defend applications, Contrast employs patented, deep security instrumentation to strengthen applications before they’re deployed, protect them in production and provide visibility throughout the application lifecycle. As a result, organizations can act faster against threats and immediately reduce their attack surface. More information on Contrast Security can be found at


# # #

For more information:
Mark Hodgson | Contrast Security | +1-650-270-4426

Contrast Marketing

Contrast Marketing