Of course you're right, Alex. There's no way that racism against white people is any more okay than racism against anyone else.
I was in a racial minority at high school. I'm a white girl and I was surrounded by Asians. Most of these girls were very cool about it, and it actually ended up being a fantastic experience, since I got to engage in things like festivals with them and I learnt a lot about their various cultures. But there was one girl who was, I don't know, resentful that I was friends with a lot of Asians? When I should have stuck with the other white girls? Idk.
Anyway, she was horrible to me for most of high school. At various points she spread rumours about me (after the first couple, everyone ignored her) and generally treated me pretty poorly. It hurt. So yeah, I got a taste of why racism isn't okay, no matter who it's directed to, of from whom. I was just lucky that the rest of my Asian friends were so supportive.
Having said that, I can't imagine what it must be like to have to put up with that when living as a minority for your entire life. I can kind of understand why it's seen as "worse" for a white person to do it, and it has nothing to do with history - white people are at an advantage to begin with, and rather than starting a problem for the other races, chances are they're compounding one. On the other hand, if someone is racist toward a white person, it's more than likely only adding to what self-inflicted white guilt has achieved.
That doesn't make the individual act of racism against white people any less horrible than vice versa. It just makes the overall effect less profound.