The series of missteps and failures that led to a Turkish government-related agency eventually ending up with a valid wild card certificate for Google domains began in June 2011 when the TURKTRUST certificate authority began preparing for an audit of its systems and started moving some certificate profiles from production systems to test systems. Two months later, a pair of subordinate certificates–which carried the full power and inherited trust of TURKTRUST’s root certificate as far as most browsers were concerned–were issued, and one of them later was used by a Turkish government transportation and utility agency to create an attacker’s holy grail: a valid certificate enabling him to intercept encrypted Google traffic.
There were a lot of detours between June 2011 and the end of December, when security officials at Google discovered the fraudulent certificate for *.google.com was being used to intercept traffic intended for Google’s own servers. But the chain of events that led to that certificate ending up in the hands of an agency that works for the government of Ankara, Turkey’s capital city, is oddly free of drama and intrigue. Instead, it mostly reads like a series of bureaucratic goofs and mix-ups that led to an obscure government agency suddenly having the ability to act as a man-in-the-middle for encrypted traffic destined for a variety of Google services, including Gmail.
Officials at TURKTRUST on Thursday began describing how the incident happened, and the details are rather messy. In a post on a Mozilla group on Google Groups, a TURKTRUST official said that the issuance of the subordinate certificates was not the result of malice, but simply a mistake.
“The case occurred in August 2011 during a software migration operation, before the first successful ETSI TS 102 042 audit which took place in November 2011. The sequence of events that led to the mistaken issuance of two certificates can be best summarized as follows: Prior to June 2011, the certificate profiles on the production system were exported to the test system, containing a particular number of certificate profiles. For the sake of testing purposes, 2 more profiles were added that contain CA extensions,” Mert Ozrar of TURKTRUST, wrote in a description of the incident.
“In the meantime, the production system was also updated to meet the need of demand to contain 3 more SSL certificate profiles. Hence the production system and the test system appeared to have different number of profiles by one, and the two sets matched only in the profiles at the date of the first data migration. The tests were completed before June 30, 2011. It was the unfortunate event that the production system was patched with the profiles in the test system, which had happened to contain 2 wrong profiles and lacked 3 correct profiles.
“The wrong profiles were only used on August 8, 2011 to issue those two faulty certificates which were certainly not intended to be sub-CA certificates. A certificate request on the 10th of August had called the use of one of the missing profiles, which was drawn to attention by a thrown exception. In order to fix this the whole set of certificate profiles was this time replaced to contain all correct profiles. Therefore the problem had been fixed once and for all although the unfortunate event that the certificates were mistakenly issued remained hidden.”
Once the wrongly issued certificates were in the hands of the recipients, the events get a little sketchier. One of the certificates was revoked at the request of the customer, Ozrar said. The other, which is the one issued for *.EGO.GOV.TR, the domain of a Turkish government agency, was installed on an IIS Web server to serve as a mail server. It wasn’t until Dec. 6, 2012, that the certificate was exported to a Check Point firewall and used to generate the fraudulent Google certificate.
“The Checkpoint firewall was said to be configured as MITM. It appears that the Checkpoint automatically generates MITM certificates once a CA cert was installed,” Ozrar said.
Google officials were alerted to the problem on Dec. 24 when its Chrome browser began detecting the bad certificate. Google alerted TURKTRUST to the problem, and the certificate was revoked. Google, Microsoft and Mozilla all have said they are revoking trust in one of TURKTRUST’s root certificates. But the conditions that allowed this problem to arise in the first place are still present, and likely will be for some time to come.
“It’s a problem that no one has solved,” said Matthew Green, assistant research professor in computer science at Johns Hopkins University in Maryland. “The CA system is a libertarian’s dream. There are so many places where people have to agree to do something, and it doesn’t happen. It’s backwards. We have a very bad detection network. You can commit a crime right now and no one is looking. Getting our detection network better is one way to go.”
The fundamental weaknesses in the CA system have been well-known for years, as have the various ways that attackers or malicious insiders could game that system. Researchers have demonstrated a range of practical attacks against CAs, and attackers have done the rest, with the compromises of DigiNotar, Comodo and others as clear evidence. But overhauling the system is a major international undertaking, and one in which the CAs themselves have no incentive to participate. In the meantime, experts say there’s a need to change the way browsers treat certificates and CAs.
“Subordinate certificates have long been identified as a point of weakness in the CA system. They are typically granted unconstrained power to issue certificates for any domain name. Thus, a leak of one subordinate certificate is seen as equivalent to a leak of authority equivalent to all CAs combined. Worse, subordinate certificates need not be explicitly trusted by the software that authenticates encrypted SSL connections (typically your web browser). They inherit their trust from the explicitly trusted CAs that have been vetted by your browser vendor,” Steve Schultze, associate director of the Center for Information Technology Policy at Princeton University, wrote in an analysis of the TURKTRUST incident.
“I have spent the past couple of years trying to convince Mozilla to strengthen their requirements so that CAs must disclose all subordinate certificates that they have issued, so that researchers can better map the risk landscape and so that users can make more informed trust decisions and detect unexpected subordinates. They’re getting closer, but it is taking an awfully long time.”
One proposal that’s in the works is an IETF draft by Adam Langley, Ben Laurie and Emilia Kasper of Google, which specifies the creation of a public log of all of the intermediate and end-entity certificates issued by CAs. That would enable not only domain owners but any interested party to look for mis-issued certificates.
“Anyone can submit certificates to certificate logs for public auditing, however, since certificates will not be accepted by clients unless logged, it is expected that certificate owners or their CAs will usually submit them,” the draft says.
One obstacle, however, would likely be the CAs, some of whom may not want a public accounting of every certificate they sell and to whom.
“It wouldn’t be that hard to do practically, but obviously the CAs don’t want to do it for business reasons,” said Green.
