To be honest, I ask them the reason, because if it's a good reason, such as me being patronizing or talking too much or something, which is often the case with me, I try to work on it (not for them, for myself).
If it's a stupid reason though, such as because I'm white and therefore apparently automatically racist, or male and therefore apparently automatically sexist, to name a couple of examples, then I tell them to fuck off and get their head out of their arse.