I start on the highest available difficulty, simply because I usually don't bother finishing a game more than once (I take no less than two months to complete one and there's always a queue in my playlist). Also, difficulty is mostly something to get used to, I don't like when a game is too easy on me, I want to feel a challenge.
Of course, it has to be a fair challenge, I'm old enough to have played through games wholly based on Fake Difficulty, and really hate them.