The following is the full transcript of a live Threatpost chat with Charlie Miller, a vulnerability researcher at Independent Security Evaluators. During this session, Miller discussed his approach to finding security flaws, his work on fuzzing applications, his plans for this year’s Pwn2Own hacker challenge and his thoughts on improvements in Apple’s Mas OS X.
raaka: hello charlie. Can you tell us about your day to day research or your key areas of focus?
Charlie Miller: Hi. Well, some days I have more “real” work than others. On days I can do research I work on whatever I feel like. I like finding bugs, so I usually do some bug hunting, usually using fuzzing. Other interests of mine are trying to bypass DEP/ASLR. Right now I’m working on something about the problem of trying to figure out if crashes are exploitable or not. (Let me know if you want to hear more about any of this stuff).
A question submitted via Twitter: Have you ever been invited by Apple to talk to their developers about software security? Would you consider it if invited?
This is an interesting question. When I was at the speakers party at BlackHat USA last year, a security engineer at Apple that I know introduced me to his boss, who I guess is the head of product security there. I mentioned the possibility of coming out and giving a talk to Apple (i.e. I brought it up). I come from an academic background so doing this kind of thing comes natural to me. He seemed to think it was a good idea.
However, later, he wanted me to sign an NDA and I wasn’t going to be able to mention that I had ever been out there. I thought this was nuts and so declined. He was afraid I would make a big deal out of going out there and somehow use it to make them look bad, although to me, I can’t see how having a security expert visit and talk to you makes you look bad. I’d say its the opposite, when a security expert offers to come give a lecture about how attackers probe your product for problems, and you decline for an imaginary fear of bad publicity, you’ve got problems.
From @Z3r0Point on Twitter: I’m looking at grad school for Applied Math. Do you find your PhD in Math helpful in the info sec field?
Not really. My degree in math had a slight applied bent but not much. For the most part, it impresses some clients because my name says PhD and that’s about it. Perhaps it helped me think logically or something, but for the most part I don’t think it helps, despite trying very hard to think of ways I can use some of that math stuff to help me find and exploit bugs!
Mo_Effron: Is there any reason to believe Apple’s newer platforms (iTV, iPad) are any more secure than their legacy ones? Does Apple have a security program like the one Gates set up at MS?
I would hope so, just because hopefully programmers know a little more about what’s going on with security these days. I haven’t looked at iTV or iPad, but as for iPhone, when it came out it was pretty sad. But with version 2, it was actually more secure than Mac OS X for a bit since it had DEP (Data execution prevention). Of course this was to enforce the use of the app store and not for security…. AFAIK, apple has no real SDL. Check out my results from fuzzing that I’ll be talking about at CanSecWest to see proof of this.
Another one from Twitter: I saw a story on Forbes.com about your upcoming CanSecWest talk about critical vulnerabilities in Preview. Please share some more details about that presentation…
The presentation is really about fuzzing, which is a technique for finding vulnerabilities in software. I basically took the most naive approach to fuzzing and performed it against Preview/Safari, Adobe Reader, MS Powerpoint, and Open Office. The idea of the talk was to record exactly what I found and gather statistics, i.e. how many crashes do you find, how many unique crashes, how many are ‘exploitable’, etc. Almost every fuzzing talk is either ‘here is a new super way to fuzz’ or ‘I found this bug with fuzzing’, but there isn’t much out there about what you really can expect to find if you start fuzzing something.
RuntPacket: Where are some good places to start for someone interested in vulnerability research / exploit writing? books/sites/forums/etc…
Things are a little tougher than when I started. I think that’s why you see more 30-year-olds presenting at security conferences than 18-year-olds these days. Used to be if you knew stack overflows, you were in good shape. Now with ASLR and DEP, its extremely hard to get exploits working on even toy programs. Exploiting Software is a good very basic starter. I cut my teeth on Shellcoder’s Handbook which might be a little out of date now. Basically, dig in, use some fuzzers, try to get some exploits working. You have to do it to learn it.
gh0stz: Do you typically use some of the fuzzing frameworks when hunting for bugs, always roll your own fuzzers or a combination?
I usually use my own, just because I know them well 🙂 I do use Sulley at times, in fact this is what I used to find the SMS bug from last Summer.
This came in via e-mail: Many pundits have made a lot of the fact that the Mac was the first to be exploited in the Pwn2Own contest. Was the choice of the Mac as the first target because the hardware/operating system combo was more desirable as a prize than the commodity Windows laptops of the other competitors? Or was it just because Macintosh exploits occur with much less frequency than Windows exploits and would therefore be more newsworthy?
So until this year, applications on Apple were way easier to exploit than Windows. This is because Apple had weak ASLR and no DEP while Windows had full ASLR and DEP. This year, Snow Leopard has DEP, so its no longer trivial to exploit. In fact, I have lots of bugs in Safari that I easily could have exploited on Leopard but will be very difficult on Snow Leopard. So it used to be that that it was much worse, but now its mostly comparable (although still slightly behind)
DongHuo: Isn’t it true you can point a bit-flipping fuzzer at pretty much any rarely-targeted software and achieve the same results you do?
Fuzzing previously untested software will definitely find lots of bugs. However, you’d expect that if you fuzz a very good piece of software, say IE from Microsoft, bit flipping will find nothing (since they’ve already done that and found those bugs). Surprisingly though, as my CanSecWest talk show, bit flipping even finds bugs in mature software, which was a surprise to me.
Mr.Me: At what age did you start playing with computers?
You’re really going to date me, but I programmed in Basic when I was about 8 on my Atari 400.
RuntPacket: Can zero-day really happen to anyone?
Absolutely yes!
scottnl: How goes the “no more free bugs” campaign? What has the back-channel industry response been?
Ummm…. great question! I think its going good, as indicated by how many bug releases come from ZDI now. There was absolutely no back-channel industry response, no one except reporters has ever asked me about no more free bugs. I don’t think they care as long as they sell their products.
Another question from the Twittersphere: What OS/browser pairing to you use? Do you do anything special (beyond default settings) to secure yourself while browsing?
You’re not trying to pwn me are you??? Have you ever heard the saying about the cobbler’s kids not having shoes? That’s me, I’m afraid. I use Safari on OSX with no special settings. This isn’t the most secure combination, by any stretch of the imagination, but I like it. It’s designed by Apple engineers to be easy to use and ‘just work’ and it does. The risk of malware is low, and hey, I’m a security expert right 🙂 The risk of a targeted attack is real, except I don’t think I’m important enough to be targeted! So I rely on security by obscurity, I guess
macxues: if you worry so much about security of Apple products, why just not to forward all Safari bugs to them (so they can fix them)?
If I do that, how will they ever learn to find bugs on their own? I won’t always be there to protect them.
L1vingStone: How does the increasing number of Microsoft products affect the Mac platform in terms of security vulnerabilities?
I’m assuming you mean MS products running in OS X. No. Most don’t do things by default. Plus, on average code from MS is probably more securely written than code from Apple.
helloworld: What is the current state of ASLR in OS X 10.6?
I think libraries load locations are randomized (as in Leopard). I think stack, heap, executable image, and the location of dyld are not. I haven’t checked in a bit so my memory is a little fuzzy but I think this is the case.
macxues: Hi, Charlie! Which browsers are you going to attack on Pwn2Own contest? Safari as usual?
Yep, just Safari for me. Leave the rest to the smarter folks.
Another question via e-mail: Do you think anyone will be able to hack into the iPhone at Pwn2Own this year? Why? Why not?
Yes, but I’m cheating a bit. Someone I know quite well says they have an exploit for it and plan on using it. But to answer your question in a more general way, from an exploitation perspective, iPhone is no harder than OS X now that Snow Leopard has DEP. In fact it is easier because it lacks ALSR all together. (Interstingly, there was a year when iPhone had DEP and OS X didn’t and so iPhone was way harder then). These statements are true for Pwn2Own at least.
In real life iPhone is harder because you can’t just exec a shell (since there is no /bin/sh). You have to write your return oriented payload to do all your dirty work, which can be a pain. In Pwn2Own, you just have to prove you have code running, not actually do something useful, so the bar is lower. The only thing iPhone has going for it, which coincidentally is stopping me from attacking it this year, is a smaller attack surface. There isn’t as much exposed code on the iPhone. Safari for Mac OS X can do anything, render any file, etc. Not so on iPhone. There are some file types MobileSafari can’t display, some they display incompletely, and of course, iPhone lacks Java and Flash which comes by default on Safari. The easy to exploit bugs I know about happen to live in the code that Safari (on OS X) has but MobileSafari doesn’t, so no go for me.
Ben: What tools to you use for finding and exploiting bugs?
I use fuzzers (mine and Sulley) to find bugs dynamically, do source code analysis (via source navigator when source is available), and use IDA Pro for reverse engineering.
radlsneak: The appstore is full of server/file sharing apps for the iphone how do you feel about finding vulnerabilities in them and developing exploits for them?
I’m not real interested in vulns in appstore apps. First, they’re not super widespread so the affect is not huge. Next, they operate in a pretty strict sandbox, so you’d either have to break out of that or you wouldn’t be able to do much as an attacker.
Someone sent this in from Twitter: In your opinion, what’s the most secure smartphone in commercial use today? And what’s the most secure OS/browser pairing. Please explain why...
I get these questions a lot, and I’m not entirely happy with my answer, but here it goes. Smartphones are really hard to compare since they basically all have different operating systems and security architectures. Its very hard to make a comparison between them and I hesitate to do so. For all practical purposes, they all are in the same ballpark and there isn’t much appreciable difference in their security, in my opinion. From a safety perspective, the safest is probably the one with the smallest market share since there is less likely to be malware written for it.
As for OS/browser pairing, there are two issues. One is how insecure each browser is, i.e. how many bugs does each browser have. The other is how hard is to exploit these bugs. For the first issue, it is pretty much impossible to measure, or else we would be able to fix all the bugs. If you look at my CanSecWest talk I’m giving this year (about fuzzing) you might come to the conclusion Safari is the worst in this regard, but I don’t even make that assertion myself, although its probably true. So, in my opinion, its too hard to measure which browser has the most bugs so the important thing becomes which browser is easiest (or hardest) to exploit. This is largely a function of the underlying operating system. Here, Windows is ahead of OS X since both have DEP, but OS X only has limited ASLR. As for the actual browser, you might consider Chrome since it has additional sandboxing capabilities which will make exploitation harder.
In general, you’re probably in pretty good shape to have a broswer in Windows 7. However, don’t install Flash, because at BlackHat DC, it was revealed how to defeat DEP and ALSR in Windows by using flash (so called JIT-spray techinque).
_RDN: When I look at a crash, determining exploitability usually begins with understanding the instruction at which I crashed, looking for firstly if the crash was on reading or writing memory. I consider a crash on attempting to write to bad memory as a good positive indicator of exploitability. What properties do you look for in your determinations?
Well, to really know, you have to understand everything that is going on, which is why its so hard. Perhaps something looks like a null ptr deref, but really there just happened to be a zero at some spot in memory that was read and could have been arbitrary. Perhaps in the case you mention, its a write, but you don’t control the offset of the write or the value or something. I use crashwrangler, but don’t entirely trust it. I also use valgrind, which is pretty sweet. The bottom line is (for now) you have to do it by hand if you really want to be sure. That’s what I’m trying to figure out how to change.
Another one via email: Charlie, would you expect a “2 birds with 1 stone” type of attacks to be prevalent? I.e. vulnerability in WebKit that can be exploited in both Safari and Chrome? Or would the majority of exploits target one specific platform?
Webkit is a great target because its on so many different platforms/devices. For example, my first Android exploit was a webkit vulnerability I had used previously on iPhone. However, the actual exploits will differ with the platform so it would take some work to have one bug rule them all, but its certainly possible. The same could be said for other ubiquitous pieces of softare like libz, libgif, etc.
Mr.Me: How much money do you make selling 0days?
None anymore, its forbidden by my employment contract.
JW:Do you see application whitelisting as being an improvement over traditional anti-virus suites? Or is it just a matter of time before hackers pick it apart too?
Application whitelists are really the best way to go from an AV perspective. However, I wouldn’t run a product like that. As part of my job, I have to download and run all sorts of crazy stuff and so I’d constantly be getting mad at the whitelist. This is a big difference between say, iPhone and OSX. (and iPad). I really hate the idea I can’t download anything I want and run it.
raaka: Sure. [DEP/ASLR] I‘d like to know more about your work. Have you worked on Nokia Qt -s60 or other mobile platforms?
So, for ASLR and DEP, the current big thing is the JIT-spray for WIndows. It won’t work on OSX, I don’t think, because Flash runs in a separate process. For once Apple is more secure that Windows, yippie! So I’m thinking how you could do something similar to bypass DEP on OSX, or on Windows how you could bypass ASLR and DEP without requiring Flash to be installed. It turns out JIT-spray is really slow too, which would be nice to change. I’ve looked at Android and Palm OS. A little at s60, but not much.
macxues: Have you ever written Mac OS kernel exploits?
Nope, no kernel exploits against any OS for me. That stuff is too hard 🙂
L1vingStone: What are the indicators you look for when fuzzing? And after you find a potential vulnerability, do you go the distance and attempt to exploit it?
This is hard. Its really a different skill set. I’m very good at finding bugs, but I’m mediocre at best as an exploit developer. I can get it done but I don’t particularly enjoy it and its got to be pretty straight forward.
0xbilly: RE: your response to Mr Dong. Have you ever fuzzed a browser/scripting language before?
A little bit. I can’t go into details :):):)
DongHuo: Could you provide some insight into how you attempt to track arguments and dispatch calls while reversing objective-c code? this is a major roadblock in creating reliable exploits for OS X.
I go into this a bit in the Mac Hacker’s Handbook. Honestly, almost all the bugs I find and exploits I write tend to be in code that is not heavily objective-c, so I don’t have a ton of experience in this. As for reversing it, read the chapter in Mac Hacker’s Handbook and check out something by nemo in the latest Phrack.
dennis_fisher: If you needed to compromise one machine (platform not specified) and couldn’t do it yourself, who would you choose to do the job?
Awesome question! I’d go with Mark Dowd who is the best security researcher in the world, in my opinion. After that, I’d probably choose Alex Sotirov.
TaPiOn: Hi Charlie, reverse vs fuzzer, what is the best?
Both. I’m a bit of an old schooler and like to reverse, but fuzzing is way easier and I have more luck with it especially if I’m feeling lazy. It doesn’t take mad skills to fuzz, which is another reason I like it 🙂
Mo_Effron: What’s your opinion of the quality of software being produced by Google, from a security standpoint?
I think its pretty good, haven’t looked too hard at it. I think they’re trying. Like I said earlier, its mostly the anti-exploitation technologies that are important rather than the number of bugs, which is hard to measure.
obamaram: Of your five years at the NSA — and besides that you liked it — what are you allowed to talk about publicly?
Ummm…. I went in knowing how to compile a C program and came out like I am now. You do the math on what I did there 🙂
scottnl: Can you talk about how OSX 10.6 changes things you’ve written about in The Mac Hacker’s Handbook?
DEP is the biggest one I’ve seen. Its really a game changer. The 64 bit stuff to a lesser extent. I mentioned earlier that browser plugins run in different process spaces too.
ad: How many boxes do you have that fuzz XYZ app 24/7/365?
None. I only fuzz when I’m specifically looking for something. During my CSW research kick, I fuzzed 4 products each for 3 weeks on around 3-5 machines. That was a really big effort for me but should be a drop in the bucket for a big vendor like MS, Apple, or Adobe.
RuntPacket: How effective do you think various exploit mitigation technologies are in deterring exploitation? If OSX could get real ASLR of all userland and kernel space, would your job get a substantially harder?
It’d be harder. Right now they have DEP+some ASLR. Of the executable code, which is what you really care about for DEP bypass, they randomize all but one library and the executable. So the amount of code you have is already small (so its already hard), but it’d be way harder if there was NONE – which is what full ALSR would give you.
andy: Are you planning on updating the Mac Hacker’s Handbook to cover 10.6?
Not at the current time. The publisher hasn’t asked me to and I haven’t asked them 🙂
DongHuo: RE: NSA, did you work for SIGINT or the “information assurance” directorate?
I did not work for information assurance.
Question via email: can you talk about the real prices you know that 0-days sell for in private sales these days?
Six figures
Z3r0Point: What would your recommendation be for someone who was interested into getting into the security/exploit research field? What should they learn (coding? what languages? reversing etc)? How did you learn the things you now know and use to do exploits?
Dive in and get your hands dirty. For example, figure out how the Aurora exploit worked. Take Dino and Alex’s course they’re offering. Reverse engineer some code. Try to get an internship at some consulting company. Look for bugs and write exploits, that’s the bottom line. Its hard work, but if it wasn’t, you wouldn’t want to do it 🙂
Ok, I’m out of here. Thanks for listening and not being mean to me 🙂 Much appreciated. Be good!
