Let’s talk stats: Why AppSec’s running on broken math

Let’s say your mean time to respond/remediate (MTTR) security issues is 60 days. 

Could be worse, could be better. For example, Veracode’s published figure is 298 days MTTR. Contrast’s metrics peg our MTTR at eight days. But on average, organizations do, in fact, take nearly two months to remediate critical vulnerabilities. 

But who’s counting? 

Hopefully, every organization out there is counting. Here’s why: A 60-day MTTR means that you’ve got a huge backlog. This is broken math, and it just doesn’t make for a healthy Application Security (AppSec) environment. 

An MTTR like that means you’ve got a mega backup, which means you need security experts to prioritize what to fix. They can’t discern what’s a real issue — vs. what’s a false negative or false positive — until experts dig down and figure it out. That’s not a trivial exercise, either: It can take 10 minutes (if you’re really good or really lucky), or it can take weeks. 

Typically, here’s how the sluggish process goes:

• Applications are scanned by the security team: between 4 - 7 days
• Results are analyzed by security experts: 14 days
• Tickets are created: 7 days
• Integrated into sprint cycles: 30 days
• Best scenario total: 54 - 58 days (if everyone works in sync and maintains high priority)

Error: Does not compute

Imagine that you found 10 security issues that warrant investigation. Fast-forward two weeks, and four of those 10 turn out to be wild goose chases because, in fact, the issues concerned libraries that were never called by your application. Those libraries were, simply, never in use, so there’s no way that they could have posed a threat. … but they most certainly wasted the time of those who had to investigate them. 

As of 2020, research found that, on average, 26% of security alerts were false positives. …

… But that was three years ago, and it’s just getting worse. Over recent years, attack surfaces have burgeoned. According to Vectra AI’s 2023 State of Threat Detection report, security operations center (SOC) teams on average receive 4,484 alerts daily and spend nearly three hours a day manually triaging alerts. 

Vectra AI also found that 63% of SOC teams report that the size of their attack surface has increased in the past three years. Manual triage isn’t cutting it: Surveyed security analysts report that they’re unable to deal with 67% of the daily alerts received, and that 83% of alerts are false positives and not worth their time.

Two-thirds — 66% — of surveyed SOC team members said that the number of alerts they receive is increasing, as are the associated costs..

Let’s go back to the hypothetical scenario where you have 10 security issues, but four of them were time-wasters because they concerned never-called libraries. Imagine if you hadn’t been sent on those four wild goose chases: You could have gotten to the six that mattered all that much faster. You could have gotten to them within, say, a week, as opposed to multiple weeks. 

What's to blame? 

Not to point fingers, but hey, these punishing numbers are coming from somewhere. The truth of the matter is that the market is saturated with vendors specializing in Static Application Security Testing (SAST) and Dynamic Application Security Testing (DAST) technologies (check out IDC’s in-depth report on these approaches; alternatively, here’s your TL;DR  version). 

The fundamental problems with these systems:

1. Out-of-the-box, SAST and DAST scanning tools generate inaccurate results with tons of false positives. The list is often tossed between security and development teams like a hot potato because neither has the bandwidth to validate what's real and what's noise. Besides, who wants to open hundreds or thousands of tickets?
2. Scan times can take hours or, in some cases, days to complete. The combination of extensive, noisy lists and long feedback loops causes teams to adopt a waterfall or sprint cycle approach, slowing down development.

How to fix this broken math

A quicker MTTR means that you’re shouldering a far lighter security load because you’re avoiding these ungodly numbers of alerts, of which at least 40% are false. With a shorter MTTR, you can shorten time to remediation and thus reduce security debt per application.

Unfortunately, most organizations lack an automated way to prioritize which vulnerabilities actually matter. They find themselves spending an inordinate amount of time carrying out risk rating and managing the backlog. This leaves them with little time to actually remediate the vulnerabilities — which is the whole point of finding them in the first place. 

If you can’t find the time (or staff) to remediate all the vulnerabilities you find, they start to pile up, like piles of laundry you don’t have time to wash or the moldy groceries you never got around to cooking. 

In order to lighten the heavy load of mega backlogs in your organization’s applications — backlogs that contain both known vulnerabilities and issues that may turn out to be vulnerabilities but need more time to diagnose before you can be sure if they’re real or not — you’ve got to automate the process. 

Runtime Security numbers are more rational

The most powerful approach to reduce these unworkable, unsustainable equations is to use real-time analysis and remediation through instrumentation and runtime analysis — in other words, Runtime Security. Runtime Security doesn’t just mean security in production; it's security that comes into play anytime that the software runs, whether that’s in development, testing or production.

Runtime enables development and security teams to analyze real-time data and detect vulnerabilities. This can, in fact, help organizations reduce their MTTR — say, to the zippy eight days that Contrast has managed to achieve. 

The implications include far more than just taming your feral backlog. Realtime provides observability in the form of a living, breathing security blueprint that enables monitoring of all application and application programming interface (API) activities in multiple directions, including attack surface; defenses; dangerous methods; and outbound calls to API endpoints, system interactions, database connections and file system interactions. 

Learn more

If you’re tired of working with AppSec’s broken math and want to explore how Runtime Security can help, join Jeff Williams, CTO and co-founder of Contrast, and guest speaker Janet Worthington, Forrester Research Analyst, for an in-depth discussion on how Runtime Security is revolutionizing AppSec.

They’ll explore: 

• Essential statistics and why traditional AppSec measures fall short. 
• What "Runtime Security" means for the modern enterprise.
• Trends and threats looming on the 2024 horizon and how to prepare your organization.

Yes, you can extricate your organization from the ludicrous equations that traditional AppSec has foisted on us. Join the webcast on Dec. 12. 11am PST | 2pm EST,  to find out how. 

Read more: 

• False positives + false negatives = real costs (blog)
• Learn about the hidden dangers of traditional AppSec tools and why Runtime Security is replacing them: (podcast writeup blog)
• Attack-path mapping your applications (blog)