I kind of do judge virgins, based on age. When I meet someone of either sex who's a bit older and still a virgin, it's either because they're religious (yes, they get judged for that) or they just haven't been able to form a close, romantic relationship with another human being in their lives. Perhaps that's because they just haven't met the right person, or find themselves disinterested...but it will stick in my mind. I'm talking people aged, say, 24/25 and above here. Though if I'm being totally honest it would be a little strange sleeping with a virgin of my own age also (21), just because I'd feel like I had a total upper hand.
Also, the reason that male virginity is looked down upon is because men are always shit the first time they have sex. The woman's not going to get a major kick out of it, no matter what they do. Female virgins are (or so I am reliably informed) 'tight' and interested in the whole act, and the length of said act won't depend on when they come.