The games are pretty stand alone to be honest. You won't miss anything by having not played the previous games. The games share the same world but the stories for the most part have no relation to one another and next to no characters which cross over, with the exception of the gods, if they can be called characters as they very rarely have a physical form.
I started with oblivion and found it to be great, sunk over 100 hours into it. Tried Morrowind due to recommendations and just couldn't get into it at all, the combat stunk.
Long story short. Just jump right in with Skyrim you won't be missing anything even without any prior knowledge of the series. If you feel like playing one of the others try oblivion, for all of its many, many, many faults, it is one of my favourite games.