With all of the talk around the importance of web and application security, why is there so little focus on the corporate databases, which store the most valuable data? Last week, at the annual Computer Enterprise and Investigations Conference, Threatpost had the opportunity to sit down with noted security and database expert David Litchfield to find out. During his career, Litchfield has uncovered hundreds of vulnerabilities in software from IBM, Microsoft, and Oracle. He’s perhaps best known for his database security research.
More recently, Litchfield developed V3rity, the first database forensic and breach investigation tool. According to Litchfield, this free tool enables more automated investigation of potential database security breaches. The application collects potential evidence of breaches in Oracle database data files, transaction logs, audit trails and in server memory. We thought the CEIC conference was the perfect opportunity to speak with Litchfield about his thoughts on the latest trends in database security and V3rity.
Threatpost: What sort of important trends are you seeing now in database security?
Litchfield: There’s nothing trending as new. Attackers are looking for personally identifiable information, like email addresses and names and being able to tie that obvious spear-phishing attacks. To be fair, however, I think the vast majority of database attacks are actually going unnoticed because people just don’t know what they look like. They don’t know that they’re being breached because it’s so trivial to hide these attacks.
It seems to me databases just seem to be this forgotten sphere of forensics. That really shouldn’t be. It should be first and foremost. And it’s beyond me why. I think part of the problem is there’s this no man’s land where the database guys don’t know anything about forensics and the forensics guys don’t know anything about databases, so no one’s claiming responsibility of that area. I just don’t get it. Why aren’t people doing this?
Threatpost: Sounds like a decade ago when people would say that they knew network breaches were occurring, just nobody knew how to easily spot them, or they weren’t looking.
Litchfield: Exactly that. Today, if you went to the average DBA, or incident responder, and said, “We think this database has been breached. Can you prove it one-way or the other? I think the vast majority of people would not be able to. They wouldn’t know where to look, how to look, and the tools just aren’t there, which is why I’ve obviously been doing v3rity and developing these tools to be able to give people that ability. I think currently probably 95% of database attacks are going unnoticed.
Threatpost: If you have a database that you suspect was breached, what would a DBA go look for? I imagine they’d seek any signs of anomalous access, but I’m not sure why that would be so difficult to identify.
Litchfield: You’re absolutely right. One would think it would be easy to
do, but generally it’s not. For a start, auditing is not often enabled. You do an investigation, and the first question you ask is, “Have you enabled auditing?” They often simply haven’t because it takes up a lot of resources.
Without auditing you have to start being smart about it and looking at things like the redo logs. The data files themselves that hold a wealth of information. It’s then piecing all that information together from various sources and saying, well, we can say such-and-such a person did this kind of action on such a given time, and from that we can infer such-and-such and such-and-such. A lot of it is not a beautiful audit trail that says “Jim Boggs entered into this table at such-and-such a time.” A lot of information has to be inferred if you don’t have auditing enabled. What I’m doing, with V3rity, is writing the tools to allow you to make those inferences. They’re not guesses, they’re the fragments that say someone did such-and-such.
Threatpost: What you’re describing, to me, sounds like it would be a nightmare and require a lot of analytical skills to attempt manually.
Litchfield: The thing is you can’t do it manually because all of the databases, except the open source ones, all of them are using proprietary, binary based files. So unless you know the file structures and so on, then you can’t manually investigate. What I’ve done is develop tools that enable users to get those nuggets of information.
Threatpost: What does a typical enterprise have to do when they suspect a database has been breached? Where do they start?
Litchfield: One of the things I’ll be speaking about today is the tool, V3rity. How I’ve documented this and I’ve worked out ways of getting access to this information. We can quickly run an analysis on the information. Users can spot new permissions, grants, and what has changed. If it’s a production database, for instance, no one should be granted new permissions. Let’s say someone breaks in, creates an account for themselves, and grants themselves DBA privileges. That kind of thing sticks out like a sore thumb on a production database.
Threatpost: One would think if something like that were to happen, on a production database, an alert would go off.
Litchfield: You’re right. One would think that would happen, but that requires auditing to be enabled, and again, nine times out of ten people aren’t auditing. It takes up too much time. The database administrators are all about performance, and auditing slows performance, so that’s one of the first things to get turned off. It’s a shame, but the situation’s not irrecoverable when it comes to breach investigations, because, as I said, there’s still a wealth of information there. You just need to know how to gather it.
Threatpost: How long does an analysis like this take? Contrast it.
Litchfield: You could probably do a full investigation of an average sized database within a day and have it done thoroughly as well. Whereas doing a manual investigation could take upwards of months, three or four months, just to go through it at the same level.
Threatpost: That’s why you started V3rity. Is it database forensics? Tell me a little bit about it.
Litchfield: There’s obviously breaches of database servers left, right and center, but a couple of years ago, I noticed that there weren’t actually any database forensic tools which I thought was crazy at the time. One of the most important aspects of breach investigations clearly should be the database server because that’s where the data is, and that’s what everyone’s going for. But there were no tools, no processes, and no documentation on how to do breach investigations of a database server. I’ve spent the past couple of years basically building V3rity to do just that.
Threatpost: What’s V3rity’s business model?
Litchfield: We’re not selling anything. I’m basically doing this for free. It’s just giving away free
processes and tools so people can do investigations of breached database servers, specifically Oracle at the moment.
Threatpost: What made you decide to develop these tools?
Litchfield: As you know, I started out in the vulnerability side of security, buffer overflows and stuff like that. It was only after cutting my teeth on that that I got into the database side. It’s a continuation of the work I’ve been doing for years.
Threatpost: Why do you feel the need to do this? Is it to educate and help the security community?
Litchfield: Exactly that. I got to the point where I was finding so many bugs in the Oracle database server, I came to the conclusion there’s always going to be an 0-day. The next best thing we can do is – when we do get breached – is to work out who did it, and understand how did they did it, and prevent them from doing it again. That’s how I got into the forensics side of things. Rather than trying to fix the database world by finding vulnerabilities, I’m now throwing my hands up and saying, “It’s too much a Sisyphean task for my liking. I’m just going to work out a way of finding out if someone has broken in.”
