Skyrim takes place several centuries (supposedly) after Oblivion. According to the article I read on it, the first four games were kind of the prologue arc, leading up to this. I guess it'd be kinda like skipping the introduction in a book; you don't actually need to read that part, but you probably should once you get the chance.