Are people saying things are racist or sexist or whatever too often? Ok, I'll agree with that general idea. People can and will be assholes, and it doesn't matter the sex or skin color of the person they are being an asshole to if they are being an asshole to them in the same way they would be to anyone else. Or, to put it another way, if the way you insult someone is the same regardless, it isn't racist or sexist, it is just being an asshole.
Do insults based relying on reference to race or sex result in racism or sexism? Well, that is harder to explore. On one hand, it is a custom tailored insult based on race or gender, so in that regard, while it isn't discrimination because of race or gender, it is negativity that applies race or gender. On the other, it is using an aspect of the person in reference or in whole as a means of an insult, and comes off as no more powerful then mentions of height, weight, glasses, whatever else. To say a mention of gender in an insult is sexist is to support the idea that a mention of height is heightest. Once you apply the logic to anything but race or gender or sexual orientation, this particular outlook just seems to be silly. In this regard, I think it falls on the intent of the insult, and the framing of it. Is the insult done because you represent a larger group or is it merely applying your participation as part of that group as part of the insult. Call someone a black jackass, is it a statement about race and the place of them in relation to your own or is it merely making use of an aspect of the insulted to narrow the insult to them as an individual? If the latter, I can't see that as racism, as it is treating them as an individual person comprised on many facets, an individual that they don't like and then insulting them. There is also the cultural and social terms, word depreciation and colloquialisms to note, and how different people have different definition of the same words, terms and phrases. An interesting debate, though for this part, I still have to side that no, it isn't a racist, just still being an asshole.
Now any insult based on gender or race or whatever done because of said gender or race or whatever, or done in a manner that expresses negativity for being that race/gender rather then negativity for being an individual who was part of that (basically, you bad because you are black, not you are bad because you are a jackass who is black). Intent again seems to pop up here strongly.