To an extent it already exists (see above comment) and in FPS's and platformers it's a great idea.
But it could also beawful, just think about the first time you went into X-18 on Stalker, now think about that being even more immersive than it already was. Eep...
There could also be motion sickness issues, believe it or not I've already seen a few people beaining up in cinemas watching 3-D films. The added concentration and severity of motion in games could exacerbate that massively, or cure it as the motions matched input. But on balance it could be a great idea, especially if someone could come up with a filtering software to make old games 3-D as well.