What do you do with a hacker who’s already in jail?

Apparently, some prisoner(s) in New Hampshire compromised the prison network.

By plugging in a network cable.

It’s still unclear if any sensitive data was obtained, but the fact that an access point was established is a serious matter.

There are so many ways this could have been stopped.

In the first place, the network switches should disallow rogue devices. Plugging in an unexpected device should at least fail to work, if not send an immediate alert to the systems administrators. They would find out which port had the access attempt, and be able to find exactly where the inmates were attempting to connect.

Now if you didn’t stop the improper connection at the switch, you could have controls in your DHCP server that wouldn’t provide an IP address on the network, and/or firewall rules to restrict access to known machines, and/or an IPS (intrusion prevention system) on the network.

If you managed to let the connection through anyway, you can run network baseline checks to catch something like this. I’ve previously talked about baselining in regards to an operating system’s profile, but the same principle applies to networks as well.
I’ve even got some sample commands (replace the loopback IP with the network range you want to test.  192.168.1.1-254 for example).

nmap -sV -T4 -O -F --version-light 127.0.0.1 -oX C:\test.xml
ndiff --text C:\baseline.xml C:\test.xml > C:\results.txt

This example uses Nmap to compare the current network state to an established baseline (created the same way as the first line, just a different filename). Nmap has a ton of features for examining networks, including Ndiff which compares two Nmap scans. This is highly useful, since in addition to being able to detect rogue devices, you can also detect when the systems that are supposed to be on the network change in some way. Perhaps a security patch that was applied, or a new piece of software opened up a new port. Or more insidiously, an insider threat has set up a personal backdoor.
The two commands above can be set up as a scheduled task in a windows environment, or as a cron job in *nix or OS X. I like to follow it with another job that emails the result file to me.

SSL revisted. (Bonus item: Baselining)

Firstly, I want to revisit my recent post on the issues with SSL and the trust model currently in use.

The ‘ComodoHacker’ has struck again this week, compromising DigiNotar and possibly GlobalSign.  The Certificate Authority trust model is beyond broken and something has to change.

ThreatPost has a great summary of the problems.

Completely separate topic, but I want to talk about the importance of baselining systems.

When you have your production system running exactly the way want it to, take a quick baseline.

In Windows, run “tasklist /svc” from the command line.  This will give you all running processes and any services that are associated under them.  To be useful as a baseline this should be output into a file.  For example:

C:\baseline\tasklist /svc /fo csv > baseline.csv

Now we have the list as a csv file in the C:\baseline folder.  This can be simply opened in Excel or imported into a database.

Weeks or months down the line, when your system starts acting strangely, run “tasklist /svc” again, and compare to your baseline.

Are there any new processes or services running?  What are they, and why are they on there now?

A good baseline will help identify when your system has been compromised (either by malicious attackers, or an internal user who didn’t mean to cause any issues).

In order to maintain the integrity of your original baseline, it would ideally be stored on a flash drive and kept in a locked drawer.  If someone had gained access to your system and noticed your baseline file, they might try to change it to mask their presence.   Storing a hash of the baseline file will allow you to verify that it hasn’t been altered.

Alternatively, the baseline could be kept in a database as mentioned above.  There are a number of benefits to this, including versioning (to keep records of all approved changes to the baseline) and better security (provided you secure the database properly).  However, if your systems have been breached, the possibility exists that the database could also be altered.

Notes:

  • To get a Linux baseline the most basic command that should work for all distros is  “ps -aux”.
  • This is about getting a very quick and easy baseline.  For very important systems, you would want do a more in-depth look at the system.

Good News! A new worm!

It’s been a while since a worm has made the rounds of the Internet (I think Stuxnet may have been the last one to make any waves).

Morto is the new kid on the block.

Although my title is meant to be sarcastic, there is actually good news about this worm.  It spreads by attempting to log into an open RDP (Remote Desktop Protocol) port, passing the Administrator user account and using a dictionary-based password crack.

The good news is that OS security has improved so much over the past few years that malicious code now has to rely on bad practices by administrators, rather than using exploits in running processes.

Of course the bad news is that any systems were vulnerable to this in the first place.  Any one of three changes in security practice stop this worm from getting into your systems.  Two of them are incredibly simple.

1. Change the ‘Administrator’ name. 

In a Windows environment, the Administrator account is there by default and has full permissions to the machine.  The account name is not set in stone (unlike UNIX/Linux.  Root is always root).  Everyone should change their Administrator account login to something else.  It’s very simple to do, and immediately removes the threat of any scripted attack that tries to use Administrator as a login.  With just a tiny bit of extra work, you can also set up a new account named Administrator that actually has no privileges.  If an attacker is able to enumerate the users on your systems it will give them a little bit of misinformation.

2.  Enable password complexity rules.

The worm has a dictionary of common passwords that it uses to attempt to log in.  They are all terrible passwords.

Like ‘password’.

STOP DOING THAT. Systems should REQUIRE mixed-case and digits at a bare minimum.

On Windows this is one click on a checkbox in Group Policy.

3. Do not open RDP to the internet

This one is slightly more complicated, but your average administrator should be able to handle it easily.

Yes, it’s very convenient to be able to log into your stuff remotely.  I do it all the time.

The difference is that I have to tunnel my RDP session through an SSH connection.  You should block incoming RDP requests (port 3389 by default) at your firewall.

You do have a firewall, right?  I mean, I assume you do, but then again I assumed you wouldn’t have Administrator:password as your master login

Update:  Morto tries a number of common account names, not just Administrator.  Not sure if this is new information or a variant of the worm.

Adventures in systems administration

I would like to post slightly more frequently than this, but was interrupted by some major issues on some systems I work on.

I’m going to detail those problems now, since I couldn’t find much of anything on the Internet and finally solved the issue by cobbling together what bits of information I could find with a lot of trial and error.

Hopefully this will be useful to someone else in the future.

Also, this post is highly apropos today

How it all began: 

There was a login problem with a web application running on our Windows 2003 servers.  After attempting a few fixes to no avail, a reboot was suggested.

I figured it couldn’t hurt.

I figured wrong.

Upon reboot, a large number of services failed to start properly, and the system was mostly useless.  I tried changing a bunch of configuration options to restore services, rebooted again.

Now the SAM (Security Account Manager) database was corrupted and login was impossible.  Fun.

So, I restored a backup and got back to a state where all the services were jacked up, but at least I could login.

The event log was loaded with errors, but the most important one for our purposes was Event ID 333 (Microsoft should have just multiplied this by 2 and made it Event ID 666, since it indicates that your life is going to be hell.)  Here’s the message:

“An I/O operation initiated by the Registry failed unrecoverably. The Registry could not read in or write out or flush one of the files that contain the system’s image of the Registry.”

The system log was filled with this error.  Basically, nothing could be written to the Registry at all.

We. Were. Screwed.

After a lot of research, I settled on the idea that the pagefile (virtual memory space) was corrupted.  I learned that Windows loads the registry into the pagefile,  and if your pagefile goes bad….well very bad things happen.

How did I get out of this mess?

For a while, I felt like Bill Murray in Groundhog Day.  No matter what I changed, as soon as I rebooted the changes were forgotten, and I was right back where I started.  I tried to follow recommendations to wipe out and recreate the pagefile, but the file never went away. (sidenote: the pagefile is stored at the root of the drive as pagefile.sys.  However, it is hidden in such a way that the only method I found for viewing the existence and size of it is to go into a command prompt and enter  “dir /a” )

I finally had my eureka moment while messing around with the pagefile settings.   I couldn’t remove a currently used pagefile, because that command has to occur at reboot, and the command couldn’t be saved to the Registry.  But there was something I could do to immediately impact the pagefile.

I made it bigger.

The corrupted file was pegged at 8GB.  By setting a custom size to be larger, 10GB, it immediately allocated uncorrupted disk space to the pagefile which was retained after rebooting.  That gave me enough functionality to set up a separate drive dedicated to a new pagefile, and then I could completely remove the corrupted pagefile.

In addition, I set the following entry to ensure Windows would allocate enough space for the Registry:

HKLM/SYSTEM/CurrentControlSet/Control/Session Manager/MemoryManagement

set PagedPoolSize = 0xffffffff

I got my systems back online and running better than they were prior to this incident.  Hooray!