Apache Foundation Hit by Targeted XSS Attack

Combining a cross-site scripting (XSS) vulnerability with a TinyURL redirect, hackers successfully broke into the infrastructure for the open-source Apache Foundation in what is being described as a “direct, targeted attack.”

Combining a cross-site scripting (XSS) vulnerability with a TinyURL redirect, hackers successfully broke into the infrastructure for the open-source Apache Foundation in what is being described as a “direct, targeted attack.”

The hackers hit the server hosting the software that Apache.org uses to it to track issues and requests and stole passwords from all users.  The software was hosted on brutus.apache.org, a machine running Ubuntu Linux 8.04 LTS, the group said.

The passwords were encrypted on the compromised servers (SHA-512 hash) but Apache said the risk to simple passwords based on dictionary words “is quite high” and urged users to immediately rotate their passwords.  “In addition, if you logged into the Apache JIRA instance between April 6th and April 9th, you should consider the password as compromised, because the attackers changed the login form to log them,” Apache said.

Here’s what happened, in Apache.org’s own words:

On April 5th, the attackers via a compromised Slicehost server opened a new issue, INFRA-2591. This issue contained the following text:

ive got this error while browsing some projects in jira http://tinyurl.com/XXXXXXXXX [obscured]

Tinyurl is a URL redirection and shortening tool. This specific URL redirected back to the Apache instance of JIRA, at a special URL containing a cross site scripting (XSS) attack. The attack was crafted to steal the session cookie from the user logged-in to JIRA. When this issue was opened against the Infrastructure team, several of our administators clicked on the link. This compromised their sessions, including their JIRA administrator rights.

At the same time as the XSS attack, the attackers started a brute force attack against the JIRA login.jsp, attempting hundreds of thousands of password combinations.

On April 6th, one of these methods was successful. Having gained administrator privileges on a JIRA account, the attackers used this account to disable notifications for a project, and to change the path used to upload attachments. The path they chose was configured to run JSP files, and was writable by the JIRA user. They then created several new issues and uploaded attachments to them. One of these attachments was a JSP file that was used to browse and copy the filesystem. The attackers used this access to create copies of many users’ home directories and various files. They also uploaded other JSP files that gave them backdoor access to the system using the account that JIRA runs under.

By the morning of April 9th, the attackers had installed a JAR file that would collect all passwords on login and save them. They then sent password reset mails from JIRA to members of the Apache Infrastructure team. These team members, thinking that JIRA had encountered an innocent bug, logged in using the temporary password sent in the mail, then changed the passwords on their accounts back to their usual passwords.

Then the attack spread to Bugzilla:

The group said that one of the hijacked passwords was the same as the password to a local user account on brutus.apache.org that had full sudo access. The attackers were thereby able to login to brutus.apache.org, and gain full root access to the machine. This machine hosted the Apache installs of JIRA, Confluence, and Bugzilla.

Once they had root on brutus.apache.org, the attackers found that several users had cached Subversion authentication credentials, and used these passwords to log in to minotaur.apache.org (aka people.apache.org), our main shell server. On minotaur, they were unable to escalate privileges with the compromised accounts.

About 6 hours after they started resetting passwords, we noticed the attackers and began shutting down services. We notified Atlassian of the previously unreported XSS attack in JIRA and contacted SliceHost. Atlassian was responsive. Unfortunately, SliceHost did nothing and 2 days later, the very same virtual host (slice) attacked Atlassian directly.

 [ SEE: Apache Site Hacked Through SSH Key Compromise ]

Apache said the use of one-time passwords was a “lifesaver” because it limited the damage and stopped the attack from spreading to other services/hosts. “The attackers could have caused widespread damage to the ASF’s infrastructure. Fortunately, in this case, the damage was limited to rooting a single host,” it said.

However, there were some worrying security weaknesses that caused problems for Apache.  For example, the same password should not have been used for a JIRA account as was used for sudo access on the host machine.  The group also lamented the inconsistent application of one-time passwords, which were required for other machines, but not on the brutus server.

“SSH passwords should not have been enabled for login over the internet,” Apache acknowledged.

This is the second major Apache compromise in less than a year.  Last August, the main site of the Apache Foundation was hacked through an attack that used a compromised SSH key. 

Suggested articles

Discussion

  • Anonymous on

    - limit access to certain IPs for each user - (we often add this to CMS wikis etc as it is a real missing feature for web based services)

    - Personally I'd like everyone to have a card reader and issue credit card type cards - at least for the admin level accounts.

  • Anonymous on

    In other news, Apache say that their site has been running on IIS.

  • Anonymous on

    This shows that really, the infrastructure itself wasn't hacked but the end user, in this case the Apache adminitrators.  The people are always the weak point in the attack chain.  Good job to the admins though on providing such a thourough account of the attack.

  • Anonymous on

    I think the problem lies with the fact that the Apache admins were doing day-to-day stuff with admin accounts... They should have been using standard accounts and only using admin-level accounts for, well, admin-level tasks...

  • Anonymous on

    Any service that allows the masking or URI's (tinyurl) should be banned from any public posting forms.

  • Jordi Boggiano on

    In reply to the previous comment, I don't think that's a solution at all, it would be extremely easy for any attacker to set up a bogus domain with a url that looks legit and yet redirects anywhere he wants. Bashing URL shortening services is not helping.

    The only way I see that would help is to parse urls, execute them, and then if they give a new Location, replace the original url by the new location, so you are sure you display final URLs, but that is also not safe since it's easy to filter out the jira instance or whatever does the checks, and do not send the redirect in that case.

  • Brad McEvoy on

    Using cookies over plain http for authentication is dumb. It could not have succeeded if they were using http digest authentication.

  • Lava Kafle on

    Bad News for us. So, What are other possible ways to secure public sites like this one? We knew about XSS attacks many years back.

  • Brad McEvoy on

    Hi Andy,

    Quoting you: "the user gets a session cookie which is used to identify HTTP requests as coming from that user.  It is this cookie that was hijacked. " - Exactly. The user name and password is sent securely over https. That is secure. What happens after that is NOT secure, which is the point i was making. A hacker can grab your session key and they're away.

    HTTP Digest was designed to be secure for every request, not just the first. With the qop=auth extension the browser sends a different key on every request, calculated based on a strongly encrypted hash of the user name, password, a server generated random value, an incrementing value and other factors identifying the particular request. So there is no possibility of a hacker spotting your key and re-using it - the server will disallow it.

    The major factor preventing widespread adoption of this form of security it that app developers always want a HTML form, not one of those ugly browser login boxes. But thats been a no brainer for a few years now thanks to AJAX - we can have bullet proof security and pretty login forms.

  • Anonymous on

    So this is how i understand it, am I correct ?

    * the XSS attack allowed the hackers to basically do a 'replay' attack against the users (stealing their cookie)

    * HTTP Digest (even delivered via AJAX) would fix this because its not vulnerable to replay attacks 

Subscribe to our newsletter, Threatpost Today!

Get the latest breaking news delivered daily to your inbox.