I'd agree with you, OP, to be honest. I'm a white male, and I'm attracted mostly to white girls, usually redheads or brunettes (pale skin is actually quite an attraction for me, though that's more because I had fake tan girls than anything else...). I'm not usually attracted to girls of other backgrounds, although I'll admit there are some Asian and black girls who I find really attractive.
And on that note, I am by no means racist. I have plenty of friends who are black, plenty of Asian friends, plenty of white friends, and although I'm English my background is Serbian (so Eastern Europe) on my father's side.
Personally, I don't see how sexual preference can possibly be equated to the feelings of someone about a race as a whole. If I said that I didn't fancy an Afghan girl because her country is full of terrorists, then that would be racist (for the record, I don't think that and would never say or think that). If I said I didn't fancy her because she just wasn't that attractive to me personally, and that I thought an Irish girl with pale skin and red hair was more attractive, that wouldn't be racist, it would just be my own preference (my actual real life preference, as it happens), and I could easily have Afghan friends too. That's just the way things go.