From Educated Guesswork (Eric Rescorla)
The received wisdom in the security industry is that trying to qualitatively assess the security of a given piece of software is an incredibly difficult task. Some of the sharpest minds in software security–Gary McGraw, Brian Chess and Michael Howard among them–have spent years trying to nail down a framework for this task, with varying degrees of success. Not to worry, though. As Eric Rescorla writes, the government has now joined the fray with a proposal to develop standards for software security.
Rescorla, who is a well-respected voice on Internet security and system design, points out that this idea has been tried in several different forms over the years:
Now, not to say that this is totally impossible, but it’s not like it’s a straightforward matter of standardization like defining a set of screw thread gauges. The problem here is that we don’t have a meaningful model for the severity of security vulnerabilities, CVSS notwithstanding, let alone for the probability that they will be exploited. Quoting myself:
I certainly agree that it’s useful to have a common nomenclature and system for describing the characteristics of any individual vulnerability, but I’m fairly skeptical of the value of the CVSS aggregation formula. In general, it’s pretty straightforward to determine linear values for each individual axis, and all other things being equal, if you have a vulnerability A which is worse on axis X than vulnerability B, then A is worse than B. However, this only gives you a partial ordering of vulnerability severity. In order to get a complete ordering, you need some kind of model for overall severity. Building this kind of model requires some pretty serious econometrics.
CVSS does have a formula which gives you a complete ordering but the paper doesn’t contain any real explanation for where that formula comes from. The weighting factors are pretty obviously anchor points (.25, .333, .5) so I’m guessing they were chosen by hand rather than by some kind of regression model. It’s not clear, at least to me, why one would want this particular formula and weighting factors rather than some other ad hoc aggregation function or just someone’s subjective assessment.
Stay tuned, as this likely won’t be the last we hear from Rescorla on this subject.
