Oddly enough, Half-Life 2: Episode 2.
The game was going great until the final part of White Forest. That part somehow managed to combine all the things I hate and dread in games. Time limits, vehicle sections, navigation with a time limit...
And then of course the whole accuracy thing with little room for error, as in running over the Hunters with the car, which was like steering a surfing elephant with 1 leg. And then throwing the Magnusson Device (WHY NO ROCKETS?). I tried it 3 times, failed, and that was years ago. Probably won't try it again.
It's not necessarily the challenge itself that's a problem for me, it's the clusterfuck combo of things I personally find tedious and annoying in games, there's just no potential for fun.