I find I have less of a problem with 'special' people and more of a problem with their carers.
When I was at school there were a few kids in the year that had autism of varying severity. One or two had quite severe autism and were obviously very anxious socially and found it difficult to learn, and they were very dependent on the one-to-one care they received, on the other hand, a couple of the kids had seemingly minor autism and could easily hold conversations with people and were fine studying in the subjects they enjoyed. The problem I had was that the carers treated all of the autistic children in the same way, as if they were incredibly needy and couldn't handle anything on their own, I remember thinking at the time that, with the more able kids, if they had just been treated the same as other 'normal' kids, then their autism would not have had anywhere near the same impact on their school life as it was having, all just because their carers essentially treated them like very young children, rather than the teenagers that they were.
Another situation I remember was going to see a movie, and there was a group of special-needs adults who were in the cinema, along with their carers. One of the people was constantly groaning, shouting and basically ruined the film for me. Now I'm all for inclusiveness, however in this case, I don't think the guy was even really capable of understanding what was going on in the movie, so why was he there? I can't help but feel like it was just the carers who wanted to see the movie, and the only time they had was whilst they were at work, so they just brought along these disabled people to kill two birds with one stone, and it just pissed me off.
Anyway, long story short, I think I just hate people, indiscriminately.