It depends on the nature of of the insult. If it's of the "Hey ******" variety, I'm well-equipped to bear those.
I'm sort of a physically intimidating person to most (6'3" and 225lbs), plus I don't really wear my emotions (people think I'm cold), so when someone insults me, I don't crack a smile, I don't look angry, and I don't respond immediately. What I do is make eye contact with the person, walk over to him, get to an uncomfortably close range, and say "I'm sorry, I couldn't hear that. Can you pull the dick out of your mouth and repeat it please?"
Eye contact is usually enough to get a person to stop any whispering behind your back, a more direct confrontation/response will be enough to stop any more vocal insults although it may be more uncomfortable (but which is worse, really; being uncomfortable and insulted or being uncomfortable after actively standing up for yourself?). If it's more personal than that, then you have to make a choice; either tolerate it and thicken your skin or get comfortable with the fact that at the worst you will get punched in the face (which is usually just more surprising than painful in my experience) and launch your own barbs at the fucker. I would usually try to spin the other person's insults in your favor. Make him seem like a hypocrite or use logic to make his insults apply to him as well. This might take a little practice to become effective as it usually requires some on-the-fly creativity.
Usually, people will insult you in order to assert some sort of dominance; they can't or don't want to fight you. Brushing them off will only give them the impression that doing it will be free of any consequence and they will assert the dominance you allow them to have whenever they have an opportunity. That is, unless the person genuinely likes it or he really wants to fight and for some reason needs you to give him any reason (by, you know, existing), in which case he will be unusually aggressive. That's really the best case to just walk away. So, use your best judgment and don't get yourself into some shit that you're not prepared for. If you're not comfortable with taking some punches then just walk away.