Frustration mostly, and there are a select few things which annoy me... though that probably goes for just about everyone (except naming specifics). However, I start off by saying that for a reason: I'm typically not annoyed by certain things automatically (they're given a fair shot to try and it do it well first). Now, determining exactly what has a high probability of frustrating and/or annoying me is a bit more interesting.
Escort Missions
A classic, but there can be such a thing as a good escort mission... the trick is to not be guiding a suicidal or fragile NPC. If the escortee can hold their own, good; if they're immortal, even better; if they kick more ass than the player could ever hope to (who's escorting whom here?), do it. However, the common escort mission involving the suicidal wet noodle NPC is almost guaranteed to piss most players off.
Stupid and/or bad writing which takes itself seriously
Games don't have to be serious, mature, or intelligent all the time; sometimes even bad writing can be done in such a way that goes all the way back around to good, or "So Bad It's Good [http://tvtropes.org/pmwiki/pmwiki.php/Main/ptitlethsq3mffp09i]". Typically this type of work has to be so absurd it's a parody of itself, but it can work (though it may require you to be right frame of mind to enjoy). Bayonetta is a good example of such a work, it revels in it's own absurdity and over-the-top everything that it's one giant joke. However, this concept can very easily go the other way and straight into horrible territory; taking itself seriously is usually the main error. If you're going to be stupid with the writing, be silly; if you're going to be serious, avoid bad writing at ALL costs.
Absurd Difficulty Spikes
Most of the time, how challenging/difficult a game isn't measured in absolute terms. I'm more inclined to say it's moreso due to how quickly the relative difficulty increases within the game itself. You can ramp up the challenge (almost) as high as you like if you if you keep it at a pace the player can continually handle. Then we have the difficulty spikes. Most games should probably have one or two of these, usually devised as a "test" of sorts for the player to see if they've learned enough of the mechanics already presented to them. However, should the spike be too large, it's more frustrating than a fun challenge. Getting the difficulty curve of a game right is essential, as screwing it up will result in a boring or frustrating game (both of which alienate the player). The thing is... avoiding absurd spikes doesn't mean hand-holding, it just means giving the player the knowledge and practice (disguised as normal gameplay) they need to defeat the spike by the time it comes around. This is why some games these days get praised for being "tough, but fair" (a term I've don't recall hearing before the current generation), they're games which are a fair bit more challenging than the standard seen these days... but still managed to get the difficulty curve right.
Online Gaming
An umbrella term, but for the most part I've noticed that online gaming is not for me... and it's predominantly because of the communities rather than the games themselves (though after a five year stint of WoW, I have a good reason to avoid endless games). I just don't fit into the hyper-competitive, rude,obnoxious, and otherwise idiotic communities which dominate online gaming; I'm more inclined to play for fun and/or relaxation, not spout a ceaseless stream of obscenities to look "cool". And quite frankly, it's typically not children who are guilty of that sort of behaviour; it's adults who behave like children.