There is nothing inherently wrong with QTE's. You could argue that the entire video game is just "pressing the right button at the right time", the only difference being that in QTE's you're getting told which one.
Using them occasionally in non-critical situations is what I think would be best. For instance, in Wet you can only kill mini-bosses by QTE's, no matter how much ammo you pour into them. This is a bad use of the Quick Time Event, because it's stupid and pointless.
The whole combat mechanic in Prince of Persia was essentially a QTE, but it worked quite well because it allowed you to mix it up and create your own combos out of them. You could press any button to create a different effect.
Dieing because you didn't press X at the right time is frustrating. Maybe just losing a bit of health, or having to go through a longer route through the level because you mis-timed a jump, would be a more appropriate penalty.