The metrics movement that has been slowly but surely infiltrating the security community in the last few years has had its own annual gathering–Metricon–for some time now. It’s been a small, quasi-academic conference since its inception, but now Metricon’s organizers are branching out a bit, bringing in some speakers and content that are outside the norm and appealing to a broader audience.
By design or happenstance, the security metrics community has mostly been an insular one, with a small group of enthusiastic participants debating the value of various methods for measuring the effectiveness of security controls or programs. There was disagreement, there was innovation, there was little consensus. And it was good.
It got even better when Metricon hit the security world upside its head in 2006, putting the combined brain power of Andy Jaquith, Dan Geer, Adam Shostack, Gunnar Peterson, Betsy Nichols and Pete Lindstrom to work on the idea of a conference that would help define and shape the metrics conversation. Jaquith, who actually wrote the book on security metrics in 2007, Geer, Shostack and the rest worked hard to make the idea of metrics appealing and palatable to the rest of the community, and they’ve succeeded to a large degree.
Now, the challenge is making metrics–and talks about the topic–entertaining and interesting enough to get rank and file security folks involved in the effort and looking at what the outcomes of their security programs are and how they can be improved, rather than how many alerts a given IDS threw off in the past 24 hours.
Metricon 6, which is Aug. 9 in San Francisco in conjunction with the USENIX Security Symposium, is the first attempt to make that happen.
“It’s more of an actual conference where you’ll talk to former CSOs and people like me and Jake Kouns and Wendy Nather and have access to real data from real research, rather than what’s happened in the past sometimes with people talking about IDS data for correlation that no one is ever going to use,” said Alex Hutton, the organizer of this year’s conference.
Among the speakers and topics slated for next month’s conference are a talk by Kouns, the founder of the Open Security Foundation, on cyberinsurance, a presentation on the effects of insider threats from US-CERT and a talk from Nather of the 451 Group on risk.
“Everybody believes that security is way too important to describe it using adjectives and adverbs. The obvious thing you want to do is give objective numbers. Even subjective numbers would help in some cases,” Hutton said. “The goal this year is to say, let’s move beyond the color scale, management by Crayola color. Let’s really talk about how we can use metrics in a way that matters, and not just as an academic pursuit.
“The opportunity this year is to talk to people who have skinned their knees in this field and are some of the best risk analysts out there,” Hutton said. “People who know where metrics fit and where they don’t fit.”
