I don't mind amnesia when done right. PS:T was done right. Often it's not... I can't actually think of a recent game that's used it, though, so there's not much to annoy there. Are there a bunch of recent games that have used amnesia, then, or are we all still annoyed at the SNES Final Fantasies?
The Ultima Series -- speaking of old games -- really pulled off the 'Chosen One' idea during its middle trilogy. Having a game where you MADE yourself the Chosen One, followed by games where people refused to believe you were the Chosen one (and weren't later convinced by your deeds of daring do) where being the Chosen one means you accidentally destroyed innocent worlds, et cetera, et cetera ... well, to me, this worked quite well.
But in the end, what annoys me is the flavor of betrayal where the guy I'm working for was just using me all along and GASP is the final boss! Not so much that it's annoying itself, but it's just so overdone. Done well, sure, sometimes; but horribly overdone nonetheless. It just doesn't count as a plot twist any more.