Once your application security program is up and running, there are several metrics you can use to gauge your progress and optimize your program. For instance, companies typically measure their scan activity, flaw density, and policy compliance. However, very few include metrics for fix rate, despite the fact that it is an important indicator of a program’s success. Fix rate indicates how long it takes for a team to fix the vulnerabilities their scans find. Fix rate is calculated as follows:
Fix Rate = Fixed Flaws divided by (Fixed + Open Flaws)
Looking at fix rate over time measures the average velocity at which organizations are fixing flaws.
All the metrics mentioned above are important, but fix rate is especially critical. Ultimately, the most important function of an application security program effectively fixing flaws once they are discovered. In the end, you can’t scan your way to secure code.
What are the average fix rates?
For our most recent State of Software Security (SoSS) report, we analyzed the data compiled from the 700,000 scans we performed over a 12-month period between April 1, 2017 and March 31, 2018, and this reveals a pretty clear picture of the current state of fix rates.
When we look at the curve for the average fix velocity from the first day of discovery, we see that it takes organizations a troubling amount of time to address most of their flaws. One week after first discovery, organizations close out only about 15 percent of vulnerabilities. In the first month, that closure reaches just under 30 percent. By the three-month mark, organizations haven’t even made it halfway, closing only a little more than 45 percent of all flaws.
When we looked at fix rate by flaw type, we found that organizations are making a big push to fix their highest severity vulnerabilities first. Organizations managed to reach closure on 75 percent of these high-severity flaws more than 100 days sooner than the norm.
But the numbers aren’t so positive for other vulnerability rankings, such as exploitability or business criticality.
Why are fix rates important?
Speed matters when it comes to application security. The time it takes for attackers to come up with exploits for newly discovered vulnerabilities is measured in hours or days. Letting known vulnerabilities linger unfixed dramatically increases your risk. For instance, it was merely days between disclosure and exploitation of the vulnerability in the Apache Struts framework that led to the Equifax breach.
In addition, it’s important to address the most high-risk vulnerabilities the fastest. Our SoSS stats surrounding fix rate by flaw type (mentioned above) are important here. The fact that most organizations are solely focused on fixing high-severity flaws, but have troubling fix rates for flaws that are highly exploitable or business critical is problematic. Oftentimes, a low-severity flaw could be just as risky, if not more so, than a higher-severity flaw. For example, a low-severity information leakage flaw could provide just the right amount of system knowledge an attacker needs to leverage a vulnerability that might otherwise be difficult to exploit.
How can we improve our fix rate?
Here are some ways to give your fix rate a boost:
Prioritize more
Reconsider your application security policy to ensure you are taking steps to reduce your most high-risk vulnerabilities the fastest. The sheer volume of open flaws within enterprise applications is too staggering to tackle at once — which means that organizations need to find effective ways to prioritize which flaws they fix first.
For instance, not all apps are created equal, so create different requirements for different apps. An application that has IP, is public facing, and has third-party components may require all medium to very critical flaws to be fixed. A one-page temporary marketing site may only require high/very high flaws to be fixed.
In addition, consider a flaw’s exploitability, not just its severity. As noted above, some low-severity flaws could be highly exploitable, while some high-severity flaws would never be exploitable.
Scan more
This year’s State of Software Security report also revealed that those organizations that scan most frequently have the highest fix rates. Our data shows that there is a very strong correlation between how many times a year an organization scans and how quickly they address their vulnerabilities.
When apps are tested fewer than three times a year, flaws persist more than 3.5x longer than when organizations can bump that up to seven to 12 scans annually. Each step up in scan rate results in shorter and shorter flaw persistence intervals. Once organizations are scanning more than 300 times per year, they’re able to shorten flaw persistence 11.5x across the intervals compared to applications that are only scanned one to three times per year.
Prevent more
The less flaws you have to tackle, the faster you can tackle them. If developers have the secure coding skills needed to avoid introducing flaws in the first place, they will put a big dent in the work needed to fix flaws later in the cycle. But most developers have had zero training on secure coding – either in school or on the job. Our research has shown that when developers do get training or coaching on secure coding, the organization’s fix rate gets a big boost. When our customers offer eLearning on secure coding for their development team, they improve their fix rate by 19 percent. When they take advantage of remediation coaching, they improve it by a whopping 88 percent.
Learn more
There’s more to AppSec than scanning. Get details in our new eBook, Application Security: Beyond Scanning.
