Why did we write this document?
On many many mailinglists that I am on, I read topics about machine’s that might be hacked, or in any other way not reliable anymore. People give advise, and although they do their best to help, it’s not always the best solution.
I had a SANS class (Incident Handling) which supports my opinion in how to act upon a potentially abused machine. The teacher there was Arrigo Triulzi, a great italian guy who is a IDS expert.
I will not retell their story, but if you wish to learn the information as best as I did then checkout this website. It will give you courses overview etc.
Now back to the original intention of this file; I want to create a clear and easy document which one could follow when a machine is not trusted anymore.
One more note: This document is mostly looking towards an unix environment. It might also be appliable for a Windows or other OS environments, but I am not sure. I don’t know enough about those to make correct assumptions for that. If you can do that, please contact me and I will update the documentation
Last updated: 28 December 2004 by Remko Lodder (added prevention section and added some more info in the company section)
Last change was triggered by: Leo (Thanks)
I think my server might be hacked, what should i do?
There are several options you can have. Since people at home tend to care less about data etc. than a company. We will devide this piece into some sections.
Company:
If you work for a company, and the data is essential. You should first contact the local police department. They can help and assist you in what to do. Mostly the things they want you to do represent this:
Take down the current running machine, unplugging ALL cables starting with the powercable!. This way we can be sure that the data does not get altered anymore. Why do we want that? Well the software installed could be programmed to dial out when a network cable is taken out, or send an ‘email’ to the other person whom made your machine unreliable. When you disconnect the power, it can’t do much anymore huh? (dont forget UPS’es!). Although one could argue that you should savely disable the computer first it is not advised that you do this. You are advised to ‘corrupt’ data if needed. Because there might be scripts running that looks for a shutdown and then destroy any evidence. That way a clean shutdown would save data, but make it impossible to know what happened, what went wrong and thus make it more easy that you become a victim again (since you don’t know what went wrong).
Make a dedicated copy of the current disk, for example boot from a Live-CD (so that the original disk is not booted and sending information to the person whom made your system untrustable). Then use dd if=/dev/ad0 of=/dev/ad1 (for example). This copy’s every bit (even blank space) to the other harddisk. The original harddisk should be stored in a safe place and not to be touched again. Use the first backup disk to create multiple backups if neccessary. Then use tools to see what’s on the disk. The tools which you can use will be described later on.
The police might want to take the original disk, personally i think giving them the original disk isn’t that much of an issue. Make sure they sign for it (Non Disclosure Agreement and perhaps something like "The police took the harddisk and is going to investigate"). You still have the backups you investigate yourself.
Now for large companies it might be that there is a seperated department handling these kind of things. You should contact them first! They tell you what you need to do and guide you. If they feel appropriate to inform the police, they will handle that. Do not think you know better then them, large companies hire good people who are trained in working on these issues and they are much better then you (probably).
Home-user
If you are a home user then the data on the disk might not be that important as in a company. If you did have vital data on the disk, then use the same actions as listed for a company. Personally i would recommend that you use a backup disk in which you will do almost the same as above for the company. Only most of the time a personal computer does not store vital information, so calling the police might be a waste of time. (In your country the police might not be pleased if you contact them. Please check if they do handle these kind of incidents before calling them).
Again boot with an Live-CD and do forensics with the tools that will be described later on.
{mospagebreak}
Ok now we have backupped our critical systems, which software can we use to check what happened?
First of all, you should have some CD’s ready to be burned in a computer which has no access at all to the potentially hacked machine. Why? So we can be ’sure’ that the data we burn on the disc will be valid, and not tampered with.
Besides the above argument, the CD we use cannot be written upon again, so in case something goes wrong, the CD is left intact.
We shall use some different tools here, listing their names, goals, and location where you can download them and a location for more information (if different from the download location). Note that we found some extensive ISO’s that provide Live-CD Features for your machine, that have all security tools you want (Knoppix, see below). Therefor we dont put too many applications on this page.
Below is a short summary of some applications. For a complete list of allround applications checkout
http://www.forensics.nl/toolkits/
And for a complete list of tools that offer good services but are standalone checkout
http://www.forensics.nl/tools/
That host is being maintained by Jacco Tunnissen, and he did a great job on that!
- The coroners toolkit, TCT, is a commonly used tool for searching disks that had an encouter with a potential hostile person. It can read data even if it’s formated etc, untill the disk get whiped a couple of times so the active bits will be overwritten. It has the goal to let you do forensics on your computer. Though it’s a old version, below will be 2 alternatives that are more up to date then this one. You can find it here.
- Helix, is based upon Knoppix, which offers great hardware detection etc. It has been designed to especially NOT touch the host computer so that makes sure you can do proper forensic research. You can find it here.
- The Sleuth Kit, is based upon the tct commandline tools. It’s more up to date then TCT. It currently analyze’s NTFS, FAT, UFS, EXT2FS, and EXT3FS. You can find it here.
- Seekerkit, a live ISO cd that brings some securityfeatures to your machine. It has LSOF packed with it , that enables you to see what processes there are running and listening. Though that only works on the host machine, so that isn’t the most pretty option. You can find it here.
- Knoppix-std, another live ISO CD that brings full security features to your machine. It has several security tools packed inside it and they really are required if you need to do forensics. Check it out here.
Apart from the above software, there is a project which does research on these kind of activities on the internet. They are called The Honeynet Project and have a great set of documentation listed. In 2001 the project featured a forensics challenge which can be found here.
{mospagebreak}
How can i try to prevent these kind of situations?
Although preventing issues like this is not part of the scope of the document, it was requested many times so perhaps we should write something about that.
Preventing is a way to decrease the possibilty that somene hacks into your system, it does not guarentee that someone will stay outside!
If you want to prevent unauthorized changes you should install detection software like Tripwire or Aide. They can see what files changed by making a hash of the file. The hash is unique for every file and is based on the creation time, modification time, size, permissions etc. So if you change a file the hash will change. The application will notice that and generate a warning that you configured.
It is also a good idea to monitor your processes so that only regular processes are running (for example: if you have a mailserver and httpd is suddenly running you want a warning because that might indicate that someone is doing nasty tricks with your machine).
And apart from that it would be a good idea to monitor logfiles with a application like Swatch. When a sshd breakin attempt occurs the entire night every 2 seconds, it might indicate that someone is brute forcing your machine to gani access. You want to get alerted for that.
Make sure that your applications are designed with security in mind, they should not give bogus output when you press a help button and must be well tested (let someone audit the code for example). Also if you develop applications on your own, please try to document what messagse could occur, because when something else is being logged that might again indicate that someone is doing nasty tricks.
I think that with this document you have a rather complete set of ideas what you can do in case your server or personal computer might be hacked and you want to be on the safe side. Ofcourse you can boot the PC yourself and try to figure, or remove some backdoors. But then the problem arises that the machine might have more backdoors and keeps you having a untrustable computer. Also you can’t be sure that data wasn’t temperated with, so in a company perspective, perhaps they do transactions on custumer names and transfer the money to their own secret little account..
If you think i missed something huge, or small 
then contact me!
Remko Lodder 13/06/2004
