Oblivion and I have a funny relationship.
On the one hand, I find it to be an absolutely terrible game in most technical senses. The character models are ugly fish-men, the voice acting is atrocious to the point of hilarity, the combat is the most floaty non-hitting dancing in the series and probably gaming as a whole, the magic is all variations on a weak ball colored whatever element you're throwing, the persuasion system is woefully broken, the leveling of monsters and items is utterly ridiculous, the game could be instantly won in a number of ways without much effort, the archery just didn't work, a lot of the quests are tedious to the point of pain (fuck [i/]Allies For Bruma[/i]), the game can't handle more than a few enemies at a time so the massive Daedra/Imperial/whoever army is like 4 guys, the fast travel system is very poorly implemented, and the whole thing is generally buggy as hell.
And yet I can't help but love it through and through. I'm honestly not 100% sure why I enjoy it so much, but I'll try to articulate it as best I can.
The absolute key for me is the sincerity of the game. Skyrim vs Oblivion is sort of like Watchmen vs Captain America in that sense. The first is all dark and broody. Everything is happening [i/]after[/i] whatever great adventures took place. The world has gone to shit, the Dark Brotherhood has abandoned their code, the Thieves Guild is thugs, etc. It's all in the aftermath of the grand fantasy adventure. Oblivion IS the grand fantasy adventure. The Thieves Guild are ultra honorable Robin Hood types who champion the poor, the kingdom is a happy, wonderful place besieged by dark forces. No civil war, no darkness or grittiness, no Nordic racism, no Paarthanax dilemma, no moral quandaries or ambiguity, just [b/]fantasy[/b]. You're the hero, you're destined to help save the world and then you totally do it. This is partly thanks to the graphical aesthetics. Everything is covered in such vibrant, lively colors rather than Skyrim's drab and depressing snow.
The second major thing in Oblivion that was done highly well were the guilds. Each of the storylines was very interesting and almost every quest was unique and fun. This is true of most of the side missions as well as several other people mentioned. The mechanics used to complete the missions actually felt neat instead of just killing the shit out of everything.
The other part of Oblivion that was awesome as OP mentioned was the soundtrack. The tunes were just so well done and so atmospheric. The soundtrack is one of the primary reason that Oblivion is not just fun, but so damn comfy. Playing it just feels like relaxing as a little kid without a care in the world.
The final thing I want to bring up is kind of cheating, but it deserves mentioning. Shivering Isles is one of the best DLCs ever. Just sayin.
So yeah. I dislike almost every aspect of Oblivion and yet I love it and played it for ten times the amount I played Skyrim. Go figure.