Okay, if you haven't really had any experience with D&D beyond second edition, then you may want to skip 3.5 and go straight to the 4.0. A lot of people aren't too fond of the system, but I like the tactical feel of the combat. A few people have said it, and I agree completely that the 4.0 rules feel like an MMO with different combat roles and such.
It might just be me, but I can't bring myself to recommend the Red Box. For the most part, the characters and rules in that set are slightly modified, slightly simplified versions of those in the 4.0 core rules. To me, the Red Box has always seemed like Wizards' desperate attempt to attract D&D veterans who thus far have hated 4.0.
I will highly recommend Gamma World. The setting is unique, easy to get into, and a blast to play. The first boxed set comes with everything you'd need to play a game except for dice and other players, and the expansions add great new origins (races/classes) and other gameplay mechanics. The game uses a simplified version of the 4.0 rules, and it's really easy to pick up.
I can't really offer any advice on any other tabletop systems, since it's been years since I've played anything other than D20. However, I've been hearing some really good things about the Dresden Files RPG.