Depending on the content of the game I will change the settings. If it's a detailed rpg with lots of things to see and do I'll play it once on the intended difficulty. Then if I feel like replaying it, I'll turn it on hard. For Mass Effect I played the game on easy, normal, then difficult; Bioware seems to want you to do that, judging by their achievements.
For FPS and RTS I almost always start on easy mode, because I know I'll be coming back to, and playing them again. Farcry 2 was played on easy mode once, and I am sure that as soon as I brave the 15 min intro scene on rails again, I'll play it on normal.