If we weren't then we wouldn't be alive and here and now would we? Those who are more obsessed than others breed more, therefore have more children. Those children are more likely to share that obsession through genes or learning behaviour from adult role models.
Those that aren't obsessed, don't mate as much, don't breed as often. Eventually, through competition for resources (be that suitable mates, means of living or food), become extinct. Same for every animal.
If the environment changes - food runs out, space runs out, or something - and having children becomes more of a burden to survival, then those who are not as obsessed will prevail and the trend will reverse. Not really a mystery, just logic and evolution.
Why are men more aggressive about sex? Because the males in our species are stronger. Competition has encouraged aggressiveness. Were that reversed (and one day it may happen), everything will change again. We are products of our evolutionary history.