It really depends. Depends on what is more enjoyable. Sometimes, I just want to burn through and destroy everything in my sight, and kick it on the easiest setting - usually when frustrated about something unrelated.
Usually, though I prefer to play on at least middle difficulty; not because of a sesne of self-fulfilment though, but just to keep myself interested. If it's too easy it's also easily forgettable.
And of course, there's a difference between "challenging" and "just fucking with you" difficulties. Difficulty can be done "wrong", and often is...