The Inquirer-Home

Apache got hacked

Apache.org releases full report
Wed Apr 14 2010, 16:30

THE APACHE SOFTWARE FOUNDATION, which coordinates development of the world's most popular web server software, has been the victim of a rather sophisticated online attack, according to an incident report published by the group.

Earlier this month Apache tackled a targeted attack on the server that hosts its issue tracking applications. The server was running Atlassian JIRA as its issue tracker, and Apache has warned users of the Apache hosted JIRA and Bugzilla or Confluence equivalents that their hashed passwords have been compromised.

"We believe the risk to simple passwords based on dictionary words is quite high, and most users should rotate their passwords", it added, before suggesting some other possible security measures. "In addition, if you logged into the Apache JIRA instance between April 6th and April 9th, you should consider the password as compromised, because the attackers changed the login form to log them."

The problems started on April 5th when hackers used a compromised Slicehost server to add a new issue to Apache's logs. The issue contained a warning, "ive got this error while browsing some projects in jira", along with a tinyurl web link. Unfortunately the link lead users to a cross-site scripting attack, which in turn allowed hackers to steal session cookies relating to JIRA.

Concurrent to this was a brute force attack against Apache's JIRA login.jsp, in which attackers threw hundreds of thousands of password combinations at its servers. Unfortunately, in less than twenty-four hours one of these attempts was succesful in gaining entry.

"The attackers used this access to create copies of many users' home directories and various files," wrote Apache. "They also uploaded other JSP files that gave them backdoor access to the system using the account that JIRA runs under."

Following this, the attackers then installed a JAR file which was used to collect and save logins and passwords. By faking a password request email to Apache infrastructure members, the hackers were able to determine what logins would give them root access to the brutus.apache.org machine, which hosts the Apache installs of JIRA, Bugzilla and Confluence.

The Foundation reacted quickly to the attack, and began shutting down the relevant services some eight hours after the hackers began resetting passwords, and moved them to thor.apache.org. By April 13th everything was back to normal.

The detail and analysis presented in the post are enlightening, and we can only hope that other victims follow the Apache Software Foundation's lead. As it says, "We hope our disclosure has been as open as possible and true to the ASF spirit. Hopefully others can learn from our mistakes." µ

 

Share this:

Comments
On a More Esoteric Note...

I've got to commend ASF for admitting a mistake publicly, and warning us about the methodology being used to hack the accounts. Many times we all see a sort of "passing the buck" by big firms that get hacked. So...kudos to ASF. Also, I have to think that the "3 tries and you're out" password scheme can be effective, at least in slowing down brute force.

posted by : Larry, 15 April 2010 Complain about this comment
@Mike

Your advice is useful on something like a desktop computer where only one user has access at a time from a single point of entry. On a web server, I think this technique would make it trivial for anyone to launch a DoS attack, locking thousands of users out of their accounts.

posted by : Rockabye, 15 April 2010 Complain about this comment
Passwords, passwords

This still comes down to poor password discipline. LONG passwords are the best way to mitigate dictionary attacks although delays and fail2ban are also good.

posted by : Keith, 15 April 2010 Complain about this comment
Don't restrict by IP add

A better way to block a brute force password attack is to lock an account for a short period of time after a small number of failed attempts.

For instance, locking an account for 1 minute after 5 failed attempts DRASTICALLY slows down brute force attempts. Increasing the delay for subsequent failures (say up to an hour) would make brute-force attacks stop in their tracks. Hundreds of thousands of attempts would take weeks, months, or years depending on the approach taken.

A notification to admins and the user being attacked would further ensure that repeated failed attempts result in that user being disabled.

All of this takes time to develop, and isn't necessarily the top priority of developers or management until someone makes it a priority. Sadly, by the time it's a priority, it's usually too late.

posted by : Mike, 15 April 2010 Complain about this comment
Isn't it inevitable...

that tinyURL will be used to cloak compromised "real" URLs ?

It has seemed to me from the outset that any URL that doesn't easily parse into a meaningful is likely to be used as a vector for maliciousness. The average punter is barely savvy enough to read the URL in any case, when it's an inscrutable handful of characters there is no hope. Let's face it, whoever was zapped here on an Apache issue log was no naïve end user...

posted by : Fatboy, 14 April 2010 Complain about this comment
IPS

Spoofing doesn't stop an IPS or SIEM from alarming, in fact sigs are specifically written to detect such attacks. Are they not running an IPS (SNORT?) or have anything sorting/collating/reviewing logs?

Interesting....

posted by : big_slacker, 14 April 2010 Complain about this comment
@AB: Changing IPs every 3 tries would be an obstacle.

As would simply an alarm that thousands of attempts were coming from anywhere. By the way, I've already suggested using Firefox with a custom user agent string to initially validate that password attempts come from an apparently authorized source.

If you can't answer my question simply and directly, I think it a good question.

posted by : bigger_luddite, 14 April 2010 Complain about this comment
hmmmm...

I wonder... Could it have been the Chinese government again?

posted by : Layla, 14 April 2010 Complain about this comment
@ above

Do you really think those guys attacked from 1 single IP or didn't know how to spoof IP's?

posted by : AB, 14 April 2010 Complain about this comment
"threw hundreds of thousands of password combinations at its servers"

I repeat from I think the Twitter compromise story: Isn't there ANY alarm for this? Are modern systems so stupidly designed as to allow unlimited attempts, instead of ignoring an IP after three failed attempts?

posted by : bigger_luddite, 14 April 2010 Complain about this comment
aboutus
Advertisement
Subscribe to INQ newsletters
Advertisement
INQ Poll

Authorities in several countries raided Megaupload recently, shut down all of its services, seized hundreds of servers and arrested several of its executives on criminal charges.

Do you think the move was justified?