Evolutionarily speaking, the problem with polygamy (aside from an arguably unhealthy lack of diversity in the gene pool, and the often abusive nature of such set-ups) is that it adversely affects the majority of the male population. We'll even ignore the whole "most polygamous societies (Fundamentalist Mormons being the example that comes to mind most easily) either kill or exile male children in order to keep their gender ratios sufficiently imbalanced" thing.
Basically, it's just a numbers game. For every man with two wives, one man goes without a wife. So, here's how it breaks down:
Polygamy:
-Good for the most desirable men, who can really live like a pimp.
-Good for the majority of women, who can share the most desirable man, and be both supported by the wealthiest man/men, and have the "best" babies
-Bad for the most desirable women, who have to share the most desirable men.
-Bad for the majority of men, who won't get a wife at all, and thus are unlikely to reproduce
Monogamy:
-Good for the majority of men, who are likely to get at least someone to make babies with
-Good for the most desirable women, who can monopolize the most desirable men
-Bad for the most desirable men, who have to accept being limited to only one wife when they could have many
-Bad for the majority of women who have to accept marrying from the unwashed masses of undesirable men.
