The after effects of the OpenSSL heartbleed vulnerability continue to spread through the technology industry, nearly two weeks after the details of the flaw were disclosed. One of the latest repercussions is a huge increase in the number of SSL certificates being revoked, as site owners and hosting providers go through the process of replacing vulnerable certificates.
Certificate authorities and other organizations maintain certificate revocation lists that browsers can use to determine whether a certificate on a given site has been revoked. Site owners will revoke certificates for a number of reasons, including security problems. Those revocations typically go unnoticed, unless a high-profile site is involved or there’s some event that causes a large number of sites to need to replace their certificates.
Enter heartbleed.
A good portion of that increase–which saw revocations go from perhaps a few thousand a day to more than 70,000 earlier this month–can be attributed to CloudFlare replacing all of the certificates for sites that it manages. The company was one of the few organizations that got early warning about the OpenSSL bug before the software’s maintainers revealed the details.
“After learning the full extent of the bug and that it had been live on the Internet for two years, we started an investigation to see whether our private keys and those of our customers were at risk,” Nick Sullivan of CloudFlare wrote in a blog post.
“We started our investigation by attempting to see what sort of information we could get through Heartbleed. We set up a test server on a local machine and bombarded it with Heartbleed attacks, saving the blocks of memory it returned. We scanned that memory for copies of the private key and after extensive scanning, we could not find a trace of it.”
CloudFlare then issued a public challenge, asking researchers to see whether they could get the private key from a vulnerable server the company set up. Within a few hours, someone succeeded. And several other people later won the challenge, as well, retrieving the private key.
“A nagging question is why, when OpenSSL has functions to cleanse memory, are these chunks of keys being found in memory. We are continuing to investigate and if a bug is found will submit a patch for OpenSSL,” Sullivan wrote.
“The more HTTPS traffic the server serves, the more likely some of these intermediate values end up on the heap where Heartbleed can read them. Unfortunately for us, our test server was on our local machine and did not have a large amount of HTTPS traffic. This made it a lot less likely that we would find private keys in our experimental setting. The CloudFlare Challenge site was serving a lot of traffic, making key extraction more likely. There were some other tricks we learned from the winners about efficiently finding keys in memory.”
There’s been some discussion about whether it is practical for an attacker to retrieve the private key of a target server using the heartbleed attack, and Sullivan said he sees it as doable.
“Based on these findings, we believe that within two hours a dedicated attacker could retrieve a private key from a vulnerable server. Since the allocation of temporary key material is done by OpenSSL itself, and is not special to NGINX, we expect these attacks to work on different server software including non-web servers that use OpenSSL,” he wrote.
