What usually bugs me about a dubbed version is the lack of ability to pronounce words left from the original version properly, as well as a tangible lack of effort from the voice actors in many cases. By this I mostly mean names and such, especially in cases where they are using more or less actual Japanese names or words, because it sounds horrible.
Apart from that, and perhaps a bit out of the focus for the thread, the dubbed version tends to be cut "to better suit the standards of censorship and age appropriate material", which I think is just plain wrong. What is deemed fine for people in one country can't possibly actually harm people in another country.
I have always tried my best to watch or take part of something in the language it was written, to get as close to the authors intentions as possible. Of course, this is far from always possible. But when it comes to anime, is it really that much harder to read and watch at the same time, and thereby avoid shoddy voice work, needless censorship, changed music and other nuisances associated with dubbed anime?
Edit: Point I forgot to make: I think the dub/sub might very well be a cultural thing as well. Being Swedish I am used to having almost everything I watch on TV subtitled (all shows, movies etc. that are in English at least). This way reading while watching something comes very naturally, and doesn't interfere with the experience. Thus I think it is only natural to have something that is in a language that is not my own, whether I understand it or not, subtitled.