No. It's the way the human mind works. We need prejudices to help us categorise information about people and everything else in the world. When we have encountered something, we can take that information and use it in the future when we encounter someone or something similar.
Without them, our information is just a disorganised mess and we'd have to learn about each individual thing separately. We can try and limit the effect by being more aware of prejudice, but they'll always be there to some extent.
Another factor which influences this phenomenon is that humans are social creatures who will almost always divide into competing groups. Many studies, notably Sherif's study of a summer camp in 1956, have proven that in situations where people are purposely divided into groups, those groups will compete and develop rivalries. We can be altruistic to a degree, but that will often be confined to those within our own group.