Threatpost spent much of the last year chasing after Greg Hoglund, the founder and CEO of HB Gary. First, it was to get his reaction to the bruising encounter his firm had with the hacking group Anonymous. Then it was an endless series of requests on the aftermath of that hack, including the departure of HBGary Federal CEO Aaron Barr, and the company’s decision to pull out of the RSA Conference in 2011. When Greg finally did speak out it wasn’t to us.
So we were happy when Hoglund, whose firm was recently acquired by the company Mantech International Corp., agreed to speak at the Kaspersky Lab Security Analysts’ Summit in Cancun, Mexico in February. His talk there on “Lateral Movement and Other APT Interaction Patterns Within the Enterprise” reinforced Hoglund’s reputation as one of the top experts on malicious code.
Threatpost editor Paul Roberts caught up with Hoglund after the speech. And, while Anonymous and HBGary Federal were not up for discussion on the record, Hoglund offered some great insights into the delicate art of tracking down remote access trojans (or RATs) after they have a foothold in your network, as well as the mistakes companies make in trying to prevent and respond to security incidents.
Paul Roberts (Threatpost): We’re talking at the Kaspersky Lab Security Analyst Summit here. You just gave a really interesting presentation where you walked the audience through a typical sophisticated attack: what characterizes it or what are some of the things that you look for in your work and that your product looks for. Could give us a shorter version of that? What are some of the things that you guys are finding in your work and analysis?
Greg Hoglund: So the first thing we look for is any piece of software that is what we refer to as a remote access tool or ‘RAT.’ It’s just malware. The term RAT is more specific in that this piece of malware is designed to allow remote access. The attacker is outside of your network and he needs to obtain this remote connection and that’s sort of a beachhead. He can then do other things in the network.
Regardless of what the RAT is written with or how it behaves, we’re really, really, really good at detecting it without any prior knowledge or signature because of our digital DNA system.
Beyond that, the attacker will typically do things on the command line that are fully enumerated in Hacking Exposed, a book that’s been around forever. There’s nothing new under the sun.
Paul Roberts: I have a copy of it at home.
Greg Hoglund: This is interesting because, well first of all, these things are really easy to detect. If you do things on the command line you change the last access times to certain times and you leave copies of whatever you run in the pre-fetch queue. There’s all kinds of places on the system where you get forensic evidence left behind. At least in most cases, the bad guys – even the sophisticated attackers – are not cleaning up after themselves.
But I did show one example in the presentation where a much more advanced tool (that) can stop the file times, erase the logs, securely delete files and also wipe the slack space on the drive. That attacker was very sophisticated. They knew the process by which an incident response professional was going to go through (and) image the hard drive. That they were gonna use EnCase or tools like HB Gary. These things would all be subverted on the command line by the use of this tool.
That said, I still have one up on them. Digital DNA (HBGary’s core technology) would have found the remote access tool ’cause that’s sort of our secret sauce.
Paul Roberts: But it might make it more difficult to figure out where on the network they had gone?
Greg Hoglund: It would make it difficult to detect lateral movement, where they had gone, what tools were actually used. Depending on how effective such a tool is used, more or less, they may cover some of the tracks. Maybe not all the tracks ’cause actually covering up all your tracks from a forensic standpoint, can be extremely difficult.
Paul Roberts: When you’re talking about lateral movement, some folks may or may not understand what that means. What do you mean by ‘lateral movement’?
Greg Hoglund: Simply gaining access to a neighboring machine. You already have access on one machine and you have a neighboring computer in the network. This is behind the firewall. That other computer may even be in a different department. If you’re in the engineering department and that’s the accounting department, for example.
You wanna get on that other computer, so there are things that you have to do. You have to sniff a password, you have to crack hashes, you have to run tools. Many of these tools come built right in to Windows and they use a lot of those pre-built tools, like Nbtstat. Again, with the use of those tools, you’re still gonna see forensic evidence that they were used and when they were used.
Paul Roberts: So the use of these tools is a telltale sign in and of itself?
Greg Hoglund: Definitely a telltale sign.
Paul Roberts: One of the things you say is that the point of access is hardly ever the target of the attackers. That they’re almost always exploiting a weak link, so to speak, and then moving to higher value assets within the organization. What are the most common points of access for sophisticated attackers? What are the most common destinations, types of assets that they are seeking out?
Greg Hoglund: Well the point of access would be varied. There’s no hard and fast rule about that… It’s typically an end user. The end user will have opened a document that was sent to them via e-mail and that may or may not include an exploit for a vulnerability. It’s either gonna be an un-patched one or a patched one. Then typically, too, it’d be a Word document or a PDF.
That machine is now compromised and it’s your initial point of entry into the network. Typically, that machine will also connect back out onto the Internet and download a second executable, which is actually sort of a payload, if you will. That second executable has all the features of their remote access system. So that computer is now owned by the bad guy.
The point of where they’d like to go, to be honest, a lot of these cases smell like they don’t know what they’re looking for until they find it. So it’s sort of targets of opportunity. They will start exploring the network. Definitely and always a target is the domain controller. If they can gain access to the domain controller, they can crack all of the passwords for the entire domain. That’s happened on numerous occasions.
But other than that, the other targets will be the engineering department CVS (source code version control) server, anything with source code. A different group with different goals might be looking for legal documents and all the legal information.
Paul Roberts: Why legal documents?
Greg Hoglund: It depends on who they’re attacking, but the legal documents may contain a plethora of inside information about where different deals are occurring. Then the other one is – this is particular to the oil industry – looking for databases of bid lease information, which is extremely expensive to collect and it takes billions of dollars and then just gets stolen and is used by a competing oil company to basically know exactly where to bid on the ocean floor.
Paul Roberts: So rather than invest in the money to do the seismic studies and everything else, figure out how much oil is there, we spend a fraction of that hacking the organization, uploading all the data they’ve done…?
Greg Hoglund: Yeah, exactly.
Paul Roberts: I think one of the things that comes across in your presentation is that in many of the cases that you look at, the attackers are only as sophisticated as they need to be to accomplish their goal.
In other words, you say zero days are not as common as you would think. Custom tools are not as common as you’d think. Typically these attackers are using exploits for known vulnerabilities – and patched vulnerabilities.
Greg Hoglund: That’s more or less accurate. I will say that they do respond to defenses that you put up. I actually put up an example of that in my presentation, but a lot of the defenses are currently focused on the perimeter.
Paul Roberts: In the enterprise, right?
Greg Hoglund: In the enterprise. So, command and control detection: bad guys have gotten very good at coming up with new ways to do C&C (command and control) to make it very difficult for existing perimeter solutions to work. For example, they might embed the C&C within an application layer that is then vectored through a a social media network or cloud service. Google Translate is the example I gave…
Paul Roberts: That was a great one, yeah.
Greg Hoglund: There’s at least ten other sites that we’ve seen that are being used in a very similar way – essentially to proxy the data back. In each case, it was a direct result of the fact that the customers were getting very good at DNS filtering. Now it’s not really an effective strategy.
Another thing that we’ve seen is that, originally, all the C&C was in clear text. Now there are multiple layers of encryption, or certificates and the certificates are checked. So if you man-in-the-middle them, (the Trojan) actually knows you’re doing it and it doesn’t work properly or it deletes itself. Why would they do that unless the customers were sticking their man in the middle devices in there and starting to try to get into it?
So the bad guys have figured it out. Now they may have figured it out because they were losing access, but I posit that they may have also figured it out because they have key loggers on every system or that they’re reading the e-mail and they can see IT teams discussing their strategies.
Paul Roberts: So maybe when we’re talking about sophistication it isn’t merely sophistication in terms of the preparation for the attack, but also in resiliency to efforts to end the attack? Would you say that is the characteristic that separates APT from criminals hitting targets of opportunity, like with banking Trojans?
Greg Hoglund: Well there are groups that do bank heisting. They’re like APT and they’ll do all these same things. What people mostly think of when they think of malware is they think of adware and things like that, the only purpose of which is to steal PII (personally identifying information) for that criminal underground that’s monetizing it for banking fraud.
Paul Roberts: Or to own the end point for use in a botnet.
Greg Hoglund: Right. This is like the majority, statistically, of all malware infections, but as I put in my slides, they’re only one of three primary groups that are out there. The espionage is still growing and it’s gonna continue to grow as a problem. Then with the rogue hacking threats, I seriously believe we’re gonna see a cyber terrorist attack pretty soon. Someone’s gonna pop a SCADA system.
There have already been researchers who have shown that it’s possible, but they were white hats so they didn’t do anything bad. There are gonna be people who use some search engines. We know what they are to find SCADA systems and they’re gonna use that for, “Oh, I think I’m gonna pop this. Shut down the water plant” or whatever.
Paul Roberts: You’re talking about search engines like Shodan, right?
Greg Hoglund: Like Shodan, yeah, exactly. But you can run those searches yourself. You don’t even have to use a search engine. You just have to know how to write a port scanner that gets banners.
Paul Roberts: I was at the S4 conference. One of the comments there was anyone can do this and yet the companies themselves, the IT teams or the ICS companies or the customers themselves are not doing this. They’re not asking, “Are any of our systems showing up on Shodan or in an NMAP scan.”
Greg Hoglund: Security, if we actually implemented it industry wide, would do pretty good at hurting bad guys.
I’ll give you an example. There’s something called the concept of least privilege. If it’s implemented well in an enterprise a lot of times these droppers do not activate and they can’t get privilege on the system. Most companies don’t have lease privilege implemented.
Paul Roberts: So users don’t have administrative privileges on their own box.
Greg Hoglund: Right. Now there’s a reason why. If you don’t have administrative privileges on your own box, that’s a serious pain in the you-know-what. Your help desk is gonna have constant problems dealing with it. So, as it turns out, you start making concessions. You start making compromises. Pretty soon, they’re able to install printer drivers, and you’re done. I’ve seen APT go straight to the printer driver. One of the groups that we track, they man-in-the-middle HP printer drivers. They’re the attack du jour. Why? ‘Cause that device isn’t protected at the end point with a GPO (group policy object).
Paul Roberts: Even in an organization that is using lease privilege, that’s something that they’re likely to overlook.
Greg Hoglund: Yes. They’re using lease privilege on everything, but they let the end users install their own printer drivers. If they didn’t have least privilege, the APT wouldn’t have bothered with this attack ’cause they wouldn’t have had to, but see, what I’m saying is they will figure out what they have to do and only as much as they have to do to be successful.
Paul Roberts: In your experience, you cited some of the Verizon data, how long are these attacks persistent for by the time HB Gary arrives?
Greg Hoglund: Well, for us, it’s high contrast. Either an attack has just happened and everybody hair’s on fire and they parachute us in or they’ve had these problems for years and we’re coming into help them. It really is like that.
Paul Roberts: How are organizations discovering these breaches in the first place? Are they discovering them internally? Are they being informed by third parties?
Greg Hoglund: It depends. They may have a problem. It ends up on help desk. Then a little bit of investigation finds some kind of malware program. Then they have someone in their incident security team that takes a little bit of a closer look and gets nervous.
A second way is that they get a notification from the FBI that their intellectual property was just found on a command and control server somewhere. This happens more often than you’d think.
Yet another way is that a security company- and a few are here that I know of – actually focus on trying to gain access to command and control servers. If they do, they will find the information in there and then do what’s called a victim notification to the company. “Hey, you’re being targeted by this group. You may wanna know about this.” So those are three ways.
Paul Roberts: We’re reading a lot about cyber war and this nation-state sponsored espionage. So there’s competitor nations with the United States or with Western European nations that want intellectual property, want product designs to spin up their own industries based on intellectual property and investments that were made outside their country.
Is there any easy way to change that situation or to fix that situation? I imagine it is hard to just say ‘Well, we’ll just make everyone more secure!’ You’re down in the trenches with this. What is the solution?
Greg Hoglund: Unfortunately the solution’s gonna lie completely on defense at least for the next few years. We are doing a webinar – HB Gary is- on the legality of hack back, which should be pretty interesting.
The problem is that there’s not a lot you can do unfortunately. Even if you did, for example, shut down a command and control server. That won’t stop the bad guys from trying again. They’re very persistent and the human weakness is there to click on things and what not. Turns out that’ll probably work and they’ll be back in.
We also have the entrenchment problem. What if you miss something. One more access tool you didn’t know about. That’s very, very common.
Paul Roberts: You mentioned mobile devices and the coming –
Greg Hoglund: Oh, Android is the worst, too. That is gonna be probably a primary attack vector for years to come..People are going to social media. Almost every social media app is represented in some form on your mobile device. There are exploits all over the place. Android’s made the news a lot recently, but I have to tell you, iPhone’s no better. They just happen to have such a closed platform that not as many people are researching on it. All of the mobile devices potentially have these problems.
Paul Roberts: How do you hop from the device to the rest of the network?
Greg Hoglund: (The mobile device) is just a bridge between the wireless network and the 3G network and, essentially, turns into a proxy.
Paul Roberts: So own the phone and then jump from the phone to the laptop and then you’re in.
Greg Hoglund: Right. Then you’re in.
Paul Roberts: Greg, really nice talking to you.
Greg Hoglund: You, too, Paul.
-END-
