The main site of the Apache Software Foundation was compromised on Friday through an attack using a compromised SSH key, leading to concerns about the integrity of copies of the hugely popular Apache Web server, which is distributed through the Apache.org site.
Early Friday morning EDT, a message appeared on the main Apache.org site saying that the main Web server for the site had been compromised and that the foundation had taken many of its services offline as a precaution. A short time later, the foundation updated the notification, saying that the compromise was the result of a compromised SSH key, not the result of an attack against the Apache server itself.
“The Infrastructure Team of The Apache Software Foundation is currently
investigating a potential compromise of one of our servers. For security reasons most apache.org services are therefore offline, but will be restored shortly. We apologies for any inconvenience this may cause.
10:42am UTC: Compromise was due to a compromised SSH Key, not due to
any software exploits in Apache itself.
More details soon.
10:53am UTC: We have restored services on our european mirror machine which was not compromised.”
In a more detailed blog post published late Friday morning EDT, the Apache staff said that the attackers went through an account on a third-party hosting provider and were able to upload some files to an Apache server, which were then automatically synced with other machines, including the production Web server.
“To the best of our knowledge at this time, no end users were affected by this incident, and the attackers were not able to escalate their privileges on any machines. While we have no evidence that downloads were affected, users are always advised to check digital signatures where provided,” the staff wrote. “The attackers created several files in the directory containing files for www.apache.org, including several CGI scripts. These files were then rsynced to our production webservers by automated processes. At about 07:00 on August 28 2009 the attackers accessed these CGI scripts over HTTP, which spawned processes on our production web services”
Apache is by far the most widely deployed Web server and has been for a number of years. The main Apache.org is the central distribution point for the free open-source server, a fact that leads to obvious concerns about the integrity of copies of the Apache server that have been downloaded recently.
The attack against Apache is reminiscent of the incident last year in which experts warned that SSH keys generated on certain versions of Debian and Ubuntu were considered compromised because of a highly predictable random number generator. That problem led to widespread concern about the integrity of those keys.
In 2001 the Apache Software Foundation was the target of a similar attack in which an attacker was able to compromise an installation of SSH at SourceForge. When an Apache developer logged into his account on SourceForge and then used SSH to tunnel into his account at Apache.org, the attacker was able to capture the developer’s credentials for Apache.
The attacker then exploited a vulnerability in the SSH server at Apache to get root privileges and replaced the existing SSH client and server on the site with compromised versions that would log usernames and passwords. Apache discovered the intrusion fairly quickly, but had to go through an exhaustive process of verifying the integrity of the source code and binaries on the compromised servers.
