The Rise of Data-Driven Security

The phrase “you’re doing it wrong” is a common refrain in the security community these days as people wander around in various states of disillusionment with the technology and processes that have led to what many perceive as a systemic failure. But that refrain usually is not followed by any useful discussion of what’s going wrong or what can be done about it. To researcher Claudio Guarnieri, one of the major problems is obvious: we’re completely backward in the way we prioritize protection.

The phrase “you’re doing it wrong” is a common refrain in the security community these days as people wander around in various states of disillusionment with the technology and processes that have led to what many perceive as a systemic failure. But that refrain usually is not followed by any useful discussion of what’s going wrong or what can be done about it. To researcher Claudio Guarnieri, one of the major problems is obvious: we’re completely backward in the way we prioritize protection.

On any given day, the headlines are full of dire warnings about new zero-days, another bug discovered in Android or a new flaw in a major database. Inside enterprise IT departments, those bugs are simply added to the already massive pile they’ll eventually get around to patching when they have time. And often, that patching plan will be based upon one or another of the myriad vulnerability scoring systems that have emerged in the last 10 years or so.

Therein lies the problem, according to Guarnieri. Which bugs to fix first and how quickly to patch them should not be based on a CVSS score or criticality rating, but rather on how likely it is that an attacker is going to try and exploit any given vulnerability.

“We tend to be too flat and don’t take into account whether vulnerabilities are actually being exploited in the wild,” Guarnieri, a researcher at Rapid7, said in a recent interview. “It’s not efficient because there’s no context. We need to understand how bugs are being used by the bad guys. There needs to be a connection between bugs, attacks and threats. People need to understand that this kind of vulnerability is being used by this kind of attacker for this kind of attack. So then I can walk it up the chain as a high priority.”

There are thousands and thousands of vulnerabilities discovered each year now, but the vast majority of those don’t end up being used in attacks. They’re the bench players, the guys who are kept around to fill out the roster and take a beating from the big boys in practice. They just sort of hang out, like Rudy waiting for the coach to call his name, hoping that one day they’ll get in the game. But, unless it’s one of the stars–say a nice ASLR and DEP bypass bug in Internet Explorer 10–then it’s probably going to stay in the shadows and never get much run.

The CVSS (Common Vulnerability Scoring System) is a system designed to score each vulnerability based on a number of factors.

Even flaws with critical ratings may not be of much use to an attacker if they’re not in a widely deployed application. That’s one of the reasons Guarnieri believes there needs to be a major shift in the way that the industry looks at vulnerabilities in general and their place in the security chain in particular. Bringing the probability of exploitation into the equation is one step in that direction.

“Right now we’re relying on the CVSS score and broken metrics. They’re purely technical evaluations of the vulnerabilities and don’t you any absolute measurements of the likelihood of exploitation,” Guarnieri said. “For cybercriminals, Java is the main thing. It’s used for targeted attacks, but targeted intrusions come down to Office in a lot of cases. Java is the bad animal in the play for cybercrime. Knowing this gives you a lot of context and advantage when counteracting. Critical bugs are really only fifty percent of what’s being used. The rest are low and medium severity. If you filter the CVE collection down to the ones that are actually being weaponized and used, it’s a much smaller number.”

Guarnieri estimates that there are roughly 100 vulnerabilities being used or sold on the underground at any given time, and the tens of thousands of others are mostly background noise.

“That gives you a very limited context of what’s likely to happen when it comes to exploitation and helps with prioritization,” he said. “Right now, we always base security on what might possibly happen, not on what’s likely to happen.”

Guarnieri, the creator of the Cuckoo Sandbox malware analysis tool, advocates a data- and intelligence-driven approach to vulnerability analysis and security, something that’s also been espoused by others in the industry, including Dan Guido of Trail of Bits. That approach takes into account the relevance of a particular vulnerability to your specific organization, how likely it is to be exploited and what the effect would be on your organization if it was exploited. 

“People are too systematic about their security,” he said. “We’re being so exposed, it’s a disaster. Data-driven security should be the next thing. Collect and analyze the data from the wild and provide a realistic assessment of what’s going on.”

Suggested articles

Discussion

  • Conrad Constantine on

    While it's true that initial penetration and rootkit'ing only uses a handful of the available exploits, don't write off the others if they apply to systems you're running. Once a human actor is driving an intrusion, the game changes to whatever-it-takes. If the info they need to locate their final target is contained in an unofficial webapp that a handful of admins use (and don't keep updated), you can bet that it's going to be exploited. Yes, keeping up with the weaponized exploits is paramount, but don't fall into the trap of thinking that since no public exploit is known /to you/ that vulns with a smaller footprint can't be made use of. Knowing what's exposed to someone that has a foothold on the network is a good checklist to have around when things get hairy.

  • Conrad Constantine on

    On a seperate note, the move towards BI -like tools in the security realm is appealling. Infosec has for far too long been driven by "Best Practises", which invariably end up being little more than "Personal Opinion" and "Anecdotal Hyperbolic Experiences". While the sorry state of data-sharing doesn't help the scenario, the realization that  we need to start seeking the proof in the data, instead of our perceptive bias, is a good foot forward towards making security decisions that actually work, because we've got the proof that it works. We're certainly inundated with evidence of what doesn't work; perhaps we're finally on the way to balancing that scale? 

  • Anonymous on

    This is ridiculous. Scoring vulnerabilities based on context and the particularities of the  organization being assessed is something penetration testers do since day one (that's 10 years ago). Let's not forget Rapid7 was basically a 'vulnerability scanner' company that then realized penetrating testing was required and move onto that as well. Rapid7 basically is late to the penetration testing game and their inexperience is making them produce all these ridiculous statements about issues that experienced penetration testers have figured out decades ago...

    Please stop reinventing the wheel, read, study, learn, and then do the talking.. please...

     

  • Anonymous on

    I also find this article difficult to understand. Doesn’t CVSS endeavour to place a value on exploitability, this would take into consideration of the likelihood of targeted only or wormable attack being developed.

     Whilst I agree “best practice” is constructed from general consensus of opinion within the industry, it is very difficult to give put conclusive facts behind any activity and we shouldn’t be discounting the efforts of the last 15 years without a solid viable alternative – what would you propose for the average mid sized organisation? how would they be able to quantify the likelihood of any given exploit being greater for their organisation than another?    

Subscribe to our newsletter, Threatpost Today!

Get the latest breaking news delivered daily to your inbox.