To me, the problem is this:
There's a difference between a 'nice guy' and a guy who is nice.
The guy who is nice is possibly even a bit rude or blunt or quiet or broody in public, but when you get to know them a bit (and if they're actually a nice guy, it doesn't take much) you realise that even if their outward appearance isn't 'nice', they are fundamentally a nice, kind person.
These are the kind that girls want!! So it may even be possible that that 'douche' or 'jerk' that the girl you like is going out with is actually a nice person, you just have to get to know him.
Note: I am not saying that all jerks are really just teddy bears on the inside. I'm saying that the fact that this guy is dating the girl you wanted but couldn't/didn't get may naturally have skewed your perspective unfairly *against* the other guy.
The 'nice guy' is the kind of person who is polite and helpful all the time, not because it's who he really is, but because it's what he thinks will get him laid/accepted/promoted/whatever. These guys *don't* get the girls, because the girls can see right thought the 'nice' and realise that it is fake and not a genuine representation of the guy's personality. This makes a girl very wary of, say, dating such a guy, because it has creepy, contrived overtones - cf the XKCD comic! =P
Important note: I am not saying that all men fall into one category or the other. Obviously there is a continuum there. But I think it is worth pointing out that sometimes trying too hard to be nice can actually set you back, in the sense that people either find you needy and creepy, or they walk all over you. There is a difference between being inherently a nice person, and 'trying to be nice to everyone'.
A better way is to be yourself, and try not to be actively unpleasant to people!