I got an interesting problem to solve today! Top on a server was showing a process named “./john” under “nobody” owner and it was consuming most of the server CPU. Our server admin figured somebody broke into the server. And even started getting reports from other servers that others servers were being attached from this server. He then found the file was placed in “/var/tmp/mysql.sock\ ” folder and deleted the whole folder. That gave peace to the server for some time.
But then it was back again. When I heard about it, it seemed like some script junkie was playing around. Exploiting some known vulnerability in the web server (or another process) and launching this “john” process. I thought it would be “John The Ripper – the password cracking program” – but then thought the guy should be smart enough to change the name of the application.
Checked up the “.bash_history” to see if there were any trails. None. The log files did not show much. But the Top showed up a john process. We killed it. And I thought I should check the web logs – not recent ones, but something older. Since the server had good traffic, we reviewed the access and error logs. Found a suspicious IP address. Grep’d the log and found out something interesting.
There were GET and POST requests to a file called file.php. Some commands were being passed via the query string. We found the URL problematic. So we located the file, and the whole thing was clear.
Script kiddy it was.
The guy hijacked someone’s FTP password, and placed a php Shell application on the server. Since the server allowed executing system commands via php, this guy was able to upload a tar.gz onto a world writeable directory – /var/tmp was one. And then unzip the application (yes, John The Ripper), compile it, and run it from there. All through the web based console.
Smart. But not smart enough.
Cleaned up the account, the files, secured PHP, and we were back in business.
Good one hour exercise though!