If there’s one key message coming through all of the noise at the RSA Conference this week it’s the fact that there’s a pressing need for more data. Data on attacks, data on vulnerabilities, data on data breaches, data on software security, data on everything having to do with security. The mini-movement that has sprung up around metrics and measurement in security has taken over a lot of the conversation at the conference, with some interesting results.
Several different panels and talks have addressed the metrics problem from a variety of angles, with the consensus being that there just simply isn’t enough good data available in most parts of the industry. The last few years have seen a marked increase in the amount of data avilable on some topics, especially data breaches, but those are still the exceptions rather than the rule. In a panel Wednesday morning, four experts with disparate backgrounds said that a big part of the problem is that it’s not clear what should be measured or how.
Even Microsoft, which has been looking at this problem for several years, doesn’t have a clear answer. Adam Shostack, a security program manager at Microsoft, said the company has good systems in place for measuring vulnerability counts and patch counts, but is still working on how to get the most out of those numbers.
“The one thing we know is that our customer would like fewer updates and more secure software,” he said during the panel discussion, which also included Gary McGraw of Cigital, Matt Blaze of the University of Pennsylvania and Elizabeth Nichols of PlexLogic. “That’s the primary metric that we work off of.”
McGraw, who has been working on measuring software security and internal software security programs for several years, said that even the organizations doing the best job with those programs have a tough time getting the most out of their measurement efforts. But the key thing is, at least they’re doing the measurements. The vast majority of software makers and other companies that produce their own custom applications aren’t even taking that step.
“A lot of people are selling highly flammable software. There’s no one who isn’t because people don’t know how to build secure software,” Blaze said.
