I start with medium and stick with it until i finished the game once, no matter how hard it really is. The only occasion i really switched down to easy was with Street Fighter IV, since i wasn't familiar with it's mechanics, and two levels in Wipeout HD which i couldn't beat on normal no matter how hard i tried (and i regard myself as a racing game fan).
The second playthrough will then be on hard usually. Or if the game has a lot potential to try out stupid things, i switch to easy so i get faster to the area where i want to try something out.