How about this...
I despise watching sports. You sit and watch, yell at the TV when your team doesn't do what you think it should have done. If the wrong team loses, you set your couch ablaze in the street (my tribute to Buckeye fans.) Did I mention the whole drinking correlation?
I like love sports that I can be involved in. My faves are SCUBA diving, rock climbing, snowboarding, and surfing. These aren't team sports, but are sports all the same. At least two of them can be competitive...though not when I do them.
The deciding factor for me (I think) is that if I can't do it, I'm not interested. I like to be actively involved in things. This is also why, if my TV is on, I'd rather have a game in than a show or movie. I want to have something to do with what is on the screen. I think games are more active than watching TV (hence- no interest in watching sports.)
Did this make sense, or what?