I think the major reason is that a lot of those people watch the Japanese dub first, then watch the English. with that, not only the voice, but even the language, adds to characterization. When that is changed, even if the acting is quality, the characterization is changed, and people complain.
The other reason I believe (other than stupid people who just automatically think it's better in Japanese, without even hearing it in English), is that people do not actually listen to the acting, or even really the voices in the Japanese. It's like when you read a book, most you tend to give the characters a voice, and even some extra characterization, in what is read. When they hear it in English, all of the characterizations they subconsciously imagined in the subs is now replaced by what was actually supposed to be conveyed by the spoken dialogue.
Though, there is still the fact that alot are really poor, but that is also true among many of the original Japanese voices, but people don't notice because they are reading subtitles, or because they don't speak Japanese, and thus don't know what poor acting sounds like in another language.