It’s becoming evident that security practitioners have to take on a metrics mentality to improve security operations, reduce risks and better advise their critical decisions. There are several steps an organization can take to ensure that they are on the right path.
There are some must-haves that need to be in place – or at least discussed—in order for a security metrics initiative to have any chance of success:
- Collect and store all event data, even if you don’t think you need it. This is especially important since you don’t always know what you have—or what you will need—in the way of forensic data analysis.
- Discuss how metrics-minded your organization is. Make sure that everyone understands and buys in to a process competency, which is critical as you start to unearth metrics-driven results.
- Don’t reinvent the wheel – adopt a model like the one outlined below for your security intelligence process, just to ensure the basic steps are covered.
- With those baseline requirements in place, the following ten tips should help guide your efforts:
- Enroll stakeholders as early as possible. As early as possible, gather a cross-functional team of metrics-minded individuals to build the plan around collecting, analyzing, reporting, interpreting and responding to security intelligence. Engage both the experts who understand the data, and the management who will need to support changes across the organization.
- Define your event system of record – a central collection point that everyone agrees houses data they trust. It will help create confidence in the data and credibility in the results. If there is an issue with the fidelity or completeness of the data, you will spend more time arguing over that, and less time making decisions based on findings.
- Make user and asset directories a critical foundation for security intelligence. Identity and asset management systems help define the categorizations for user-specific metrics, so understand their accuracy, their ongoing refresh cycles and the user-specific “coverage” you are achieving in your measurement efforts.
- Use your IT/Service catalog to guide your metrics. Our recommendation is that you look at security policies or service level agreements – those will give you a great set of areas to contemplate building metrics around. Since you are relating what you measure to what is expected of you, the value of your results are more immediately seen by your organization.
- Establish basic measurements, understand them, then expand. Start somewhere, anywhere, to establish a metric and then work to make that metric useful or replace it with a better one that you’ve discovered in the process. Don’t just poke around or take a whack-a-mole approach to your discovery process – prioritize your effort so that you can accumulate and maintain a portfolio of metrics that maximize the value of your initiative.
- Be consistent. Don’t spend a month on analysis then move on if nothing pops up. Maintaining consistent vigilance is the key to spotting trends or variances …erratic monitoring and analysis leads to a false sense of security and reduces your ability to continuously reflect and refine based on known patterns.
- Be ready to change. There is a tendency to take a finding, create a counter-measure around it, and then never look back. Be intellectually honest when you make new discoveries, particularly if they show a need to change an established rule, alert or policy. While flexibility and change seemingly conflict with “be consistent,” get comfortable with the idea that you will often learn something new which will require a policy or process change.
- Don’t hoard. Engage experts and ignite managers. The dynamic nature of attacks may also lead you to integrate data from systems you didn’t initially consider using to drive critical correlations. As you think about what data to analyze, solicit input from teams who know the systems, devices, people or information associated with all areas of infrastructure. They may shed light on interdependencies or relationships that are critical to better metric definition. Leverage “the truth” established with the experts to ignite the support needed from managers.
- Test yourself. Conduct a Metrics Penetration Test (MPT), which determines if the analytics you have established will “catch” the behaviors you are trying to isolate. For example, have an employee download a massive amount of data from an unusual location during an odd hour of the day to see if your “Unusual Download Volume” measurement triggers the flags you expect to see. Use results from these MPTs in operational reviews to continue evolving/maturing your analytics methodologies.
- Innovate with new technologies but prune as you go. Defense in depth is a proven strategy but it can also lead to technology bloat, a false sense of protection and – in many cases – open doors for attacks. Examine your digital exhaust to identify devices, systems, applications and tools that are dormant or redundant. Much like the Meaningful Use measures practiced in the Health Care industry, security teams can really lead the way in identifying and decommissioning any hardware or software not in use since these are very easy attack targets.
There is much we are learning every day when it comes to security intelligence, and to evolve, we must adopt new disciplines around metrics management and continuous improvement. We applaud practitioners who are breaking new ground with the “science of security” and building defenses that are built around a systematic inspection of their landscape.
Joe Gottlieb is president and CEO of Sensage.
