kasia in a nutshell

Nothing to see here.. and no I'm not dead, just busy!

Posted by kasia at 10:58 PM | Permalink | Comments (2)

We've been doing a lot of phone interviewing at work lately to fill a couple positions. Granted, Connecticut isn't exactly a mecca of high tech and most CS grads flee for their lives as soon as they graduate. Even knowing that, the candidate pool still appears depressingly dismal. What really strikes me is that the worst candidates seem to be the ones with a Master's degree in Computer Science. I assume most of these are course-work only Master's. I really would like to believe that someone could not possibly complete a thesis (which in CS is usually a programming project, rather than a research paper) and not be able to answer some very simple questions pertaining to their chosen field of work.

Looking through the recent resumes two things stand out:

1. More than half of the candidates have a Master's in Computer Science.
2. Less than half (out of those only one had a Master's) did well enough in a phone screen to progress to an in-person interview.

The obvious possible conclusions are:

1. A Master's in CS dumbs a person down.
--- or ---
2. The kind of person who chooses to pursue a Master's degree is not the kind of person we'd like to hire.

The second choice leads to a conclusion that people who are not exactly the greatest in their field will move on to pursue higher degrees in order to showcase better qualifications on their resumes. Pretty much every single person we interviewed with a Master's would barely qualify for a junior-level position. It's really stunning how little interest these candidates appear to have in a field they chose to pursue graduate work in.

And here I am about to become one of them. Yikes. Pray for my brain.

Posted by kasia at 09:54 PM | Permalink | Comments (11)

Just a few books for the next couple of months
All the Mathematics You Missed But Need to Know for Graduate School
Combinatorics for Computer Science
Selected Papers on the Analysis of Algorithms
Selected Papers on Computer Languages

Saving my pennies for this expensive one..
Handbook of Theoretical Computer Science : Formal Models and Semantics

Grad school starts in two months!

Posted by kasia at 08:03 PM | Permalink | Comments (2)

A new SUV named 'patriot'. No message there!

Posted by kasia at 10:32 PM | Permalink

My server was hacked last week.

It was not a super-smart uber hacker who did it. There isn't enough interesting information on there to tempt one of those guys. It was just a script. One of undoubtedly hundreds currently perusing the net in search of easy targets. The weak point was allowing password logins with an account that uses a person's first name for a username. Once the bot is in the system it's really trivial to root a linux box, especially one running an older version of redhat like mine was.

The real kicker? I know better. I would never ever advice anyone allow password logins on a publicly available server. Why did I allow it? Convenience, laziness, thinking all the passwords should be strong enough (yah right).

I feel like an idiot (and rightfully so), but at least I spotted it rather quickly - in a couple of hours. I monitor my logs regularly and keep on top of my traffic statistics, but what really gave it away is my email stopped.

I logged onto the system to check the logs and see what's going on. The email log (/var/log/maillog on most linux systems) stopped recording about an hour earlier. That's not normal. Checked netstat to see all the usual suspects happily running and listening on all the right ports. No unusual ports opened.

This is typical of what is up on my machine:

[root@cygnus log]# netstat -an | grep LISTEN
tcp        0      0 127.0.0.1:10023             0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:3306                0.0.0.0:*                   LISTEN      
tcp        0      0 127.0.0.1:783               0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:25                  0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:26                  0.0.0.0:*                   LISTEN      
tcp        0      0 :::110                      :::*                        LISTEN      
tcp        0      0 :::143                      :::*                        LISTEN      
tcp        0      0 :::22                       :::*                        LISTEN      

At any other time I'd have thought nothing was wrong. But my maillog was not recording any traffic and that's not normal. I checked my secure log. Nothing unusual. Current logins, all normal. As far as the system was concerned all looked normal.

I checked recent logins and noticed that an hour earlier an account was used that hasn't been in a while. That's unusual, checked the IP. Romania. That's when the real alarms started going off. First check, syslog. It's not running, won't start and strace hangs. Checked /dev/ for unusual files (that's where historically hackers like to put their binaries).

 find /dev -type f -print

Nothing unusual there. Next thing to check is the date of binaries in /usr/bin and /usr/sbin. Bingo. All important binaries (netstat, lsof, etc) have a modification date of right now. That's the point where you know the system cannot be trusted. Sure, netstat is showing no unusual activity, but that's not what the system is really doing. lsof hangs, strace hangs, all the important tools that sysadmins depend on to know the state of the system can no longer be trusted.

Once a linux system is compromised like that, you can't recover without replacing the entire OS. So that's what I did. I shut the system down and asked the web hosting company to put a fresh install of whatever linux they have handy on it. As far as I can tell from the little evidence I gathered before shutting the box down, the system was compromised by a script using an account that had no sudo access. Then the box was rooted. Considering that the postfix binary was replaced and it stopped receiving external email for any of the configured domains (my postfix configuration is unorthodox in that its based in mysql), I would imagine the goal was to turn it into a spambox.

The real value to in a publicly connected linux machine is that it's trusted by other systems (until alarms go off anyway), it's not blacklisted and can serve as the perfect spambot. This is why shutting it down as soon as I confirmed it was hacked was so important.

Lack of burst in network traffic after it was compromised confirms that it was not someone interested in what information the server contained. A hacker with intentions other than turning the server into a zombie would have copied as much information as he could as fast as he could before the sysadmin was alerted to something not being kosher.

The system is back and running now, I restored my data from a backup (I have a daily rotating one) and configured it in a much smarter way. No more password based logins. Public/Private key authentication only. I had all the other important points down already, but here's the list in case it's helpful:

• No password logins, private/public key authentication only
• No root password, strictly monitored sudo access
• Strict firewall rules based on knowing what to allow and why. Deny first, allow second.
• SSH protocol 2 only, 1 is not secure
• Monitor, monitor, monitor, monitor. There's nothing that can replace a vigilant sysadmin. Tools like logwatch and mrtg are incredibly useful.
• If you think your server is hacked, shut it down. You cannot trust anything the system tells you if binaries have been replaced. It's imperative to take the system offline as soon as possible.
• Don't try to recover a hacked system, if you don't know when it was hacked, don't trust backups either.
• Password protect any private keys stored on a publicly accessible server.
• Only install application you actually use and need. The less stuff running on the system, the lower the security risk from bugs in applications.

My one reason for allowing password logins was the ability to login from anywhere into my linux box and then use my keys to login to any system I need to that requires key authentication. The solution? Generate a new, password-protected key and keep it in a usb keyfob. Use that key to login to the system.

How often do scripts target public servers? Here's some extracts from my system logs:

Over last 24 hours:

 Received disconnect:

    11: Bye Bye : 1617 Time(s)

    11: No supported authentication methods available : 1 Time(s)

    14: No supported authentication methods available : 1 Time(s)

That's 1617 of these in 24-hour period:

May 19 14:05:17 cygnus sshd[24358]: input_userauth_request: invalid user testuser

May 19 14:05:18 cygnus sshd[24358]: Received disconnect from 218.210.68.166: 11: Bye Bye

May 19 14:05:19 cygnus sshd[24359]: Invalid user tester from 218.210.68.166

May 19 14:05:19 cygnus sshd[24359]: reverse mapping checking getaddrinfo for 

adsl-218-210-68-166.pc.sparqnet.net failed - POSSIBLE BREAK-IN ATTEMPT!

May 19 14:05:19 cygnus sshd[24360]: input_userauth_request: invalid user tester

May 19 14:05:19 cygnus sshd[24360]: Received disconnect from 218.210.68.166: 11: Bye Bye

May 19 14:05:20 cygnus sshd[24361]: reverse mapping checking getaddrinfo for 

adsl-218-210-68-166.pc.sparqnet.net failed - POSSIBLE BREAK-IN ATTEMPT!

May 19 14:05:20 cygnus sshd[24362]: Received disconnect from 218.210.68.166: 11: Bye Bye

May 19 14:05:21 cygnus sshd[24363]: reverse mapping checking getaddrinfo for

adsl-218-210-68-166.pc.sparqnet.net failed - POSSIBLE BREAK-IN ATTEMPT!

May 19 14:05:21 cygnus sshd[24364]: Received disconnect from 218.210.68.166: 11: Bye Bye

May 19 14:05:23 cygnus sshd[24365]: reverse mapping checking getaddrinfo for

adsl-218-210-68-166.pc.sparqnet.net failed - POSSIBLE BREAK-IN ATTEMPT!

May 19 14:05:23 cygnus sshd[24366]: Received disconnect from 218.210.68.166: 11: Bye Bye

May 19 14:05:24 cygnus sshd[24367]: Invalid user knoppix from 218.210.68.166

May 19 14:05:24 cygnus sshd[24367]: reverse mapping checking getaddrinfo for

adsl-218-210-68-166.pc.sparqnet.net failed - POSSIBLE BREAK-IN ATTEMPT!

This is why not allowing password authentication is so important.

Posted by kasia at 06:22 PM | Permalink | Comments (11)

.. well, at least error message translation does, so users don't see this (click on image to see larger):

Arrived there from a google search, error was gone a minute later.

Posted by kasia at 11:23 PM | Permalink | Comments (2)

It's hard to believe, but three long years have passed since I purchased my old, trusty powerbook. Three, long, slow years. (Almost as much time since my last blog post, hah!).

Since it's been (a) long and (b) slow and (c) old and well, I hate having to wait 10+ minutes for a build to finish when working from home.. You can guess what I just did, right? I bought a brand-spanking new macbook pro. It's fast, it's shiny, it has a scrolling trackpad. That's not what this post is really about though.

I'm most impressed with apple's 'puter transfer tool. It feels as if I haven't really switched machines. Particularly since this one looks nearly identical to the old one. Except for that creepy camera lens staring at me from the top. It's really starting to creep me out. What if it's on and I don't realize it? What if someone out there is watching me stick the tip of my tongue out as I desperately try to make fewer typos? (Can you just visualize that? See, scary part is, maybe someone doesn't have to visualize it!). But again, that is not what this post is about.

Despite the really wonderful transfer of files, settings, browser cache, cookies and other goodies, one thing that did not move along to my new machine is my iTunes purchased music. I had to enter my password and authorize this machine.

So now.. by doing nothing other than upgrading computers, Apple is telling me I only have 3 more authorizations left before I'm no longer allowed to listen to the music i legally paid for. In other words, unless I never upgrade machines again (yah right), this music will no longer be available to me in a few years. Did I mention I paid for it legally, unlike many other users? Based on a statistic I just made up, close to 70% of iPod users don't bother paying for the music. Someone please tell me one good reason why I should, seeing as I'm being treated like a thief even when I'm trying to be good.

To hell with that. Screw you, music industry.

Posted by kasia at 08:10 PM | Permalink | Comments (8)

I have a coworker who sometimes likes to spray air freshener in the office. It really bothers me. I'm sensitive to chemicals and it always gives me a headache and causes eye irritation. Today, after another 'spraying', I had a headache, so i took some advil and went home.

When I got home I noticed there was something odd with my vision. Couldn't quite put my finger on it, but it was definitely not right. In about fives minutes, it deteriorated to serious blurriness, then it got weird. Jaggedy, colourful images dancing in front of my eyes with surrounded blurriness. I was practically blind for about 5-10 minutes and had serious spatial disorientation when I could see my surroundings. Closing eyes did not help, could still see the same hallucinations and no blackness. The feeling was very much like after a small dose of hallucinogenic drugs.

The whole episode ended in about 15 minutes and via the magic of google, I found the symptoms were very consistent with a migraine aura. I have had migraines before, but the precursor was usually blurry vision and nausea. I didn't even know migraines could cause hallucinations.

More google magic and turns out that air fresheners are pretty nasty stuff and one of the known triggers of migraines. Tomorrow, i'll have a conversation with my co-workers, but in the meantime, if you get migraines, something to think about.

Posted by kasia at 09:17 PM | Permalink | Comments (3)

While in the process of looking through code to see what I need to change for my current project I decided to clean up our somewhat bloated constants file. You know, do the thing nobody else ever does? Remove unused constants!

While removing the lonely, unused constants and getting rid of some duplicates (user_id vs userid vs user_identifier that all point to the same string.. and so on..) I decided to remove a constants that will probably return to hunt me at some point. I've decided to live with the consequences of the change. It was either change mucho code to use the constant and be (that dirty word..) consistent or remove it. So I decided to remove it. If we have to change the value in the future, it'll be a lot of work, I admit it, but I'm willing to live with that result of my actions.

Today, I removed..

public static final int ZERO = 0;

RIP, leftovers of NSBCs.

Posted by kasia at 09:03 PM | Permalink | Comments (2)

A problem we ran into at work today..

A table with mucho data and a date column. The date column has an index on it and its type is just pure old date. A query using jdbc and java.sql.Timestamp type on a prepared statement generates a full table scan.. and the associated slllloowwwwwnnnessssssss.

Identical query using java.sql.Date on a prepared statement or to_date(String, format) in the SQL uses the index.

Am I missing some obvious Oracle clue here? Cause google isn't helping!

Edit: I realize we can hint to Oracle which indexes to use, but it would be nice to have a more general solution.

Posted by kasia at 08:34 PM | Permalink | Comments (5)