I'm not sure how much of it has to do with the inability to resist. I've been known to utter something along the lines "I wish people would stop hitting on me". However, it's not because I don't feel I can resist any physical business that comes up (relatively big guy with martial arts training). It's because the attention, in my case, is 99% unwanted.
Mainly, when people "hit" on me in this fashion, it's at the bar and, typically, they're drunk. My typical experience when this happens goes something like this: My friends and I go out for some pints. We avoid dance clubs because the object of the night is to get drunk and have fun (not to hit on members of the opposite sex). Some drunk girl with a loose connection to one of my friend or myself (usually they were in one of our classes) comes over to chat a little. They then insist that I follow them to a dance club to go dancing. When I refuse, I have had them try to physically drag me away from my friends. I can easily resist, but it doesn't make their pleading and prodding any less irritating.
In reusing your rich person analogy, it's not so much like rich people walking around complaining that they have all the money in their wallet, but like them complaining that the government has taxed their money to X degree. From an outside (poor person) perspective, that X may be more than you make in a year, so people are prone to view the rich person as being greedy. However, EVERYONE would prefer not being taxed. In a similar way, EVERYONE dislikes unwanted attention. Attractive people may get MORE unwanted attention, but that doesn't make them fundamentally flawed for complaining about it.