It doesn't have much to do with sexism of the voters, but it's an issue because men (in the government) have more power on laws that affect only or mostly women (abortion, rape laws)than women do.
Edit: Yes, abortion "affects" the woman's family, too, but the only direct effect is on her.