Yeah, playing games isn't all too much different from reading books or watching TV. Except that while books can expand your vocabulary, games, and to a lesser extent TV, can do that too. At least games have the added benefit of helping you improve things like reaction time, hand-eye coordination, puzzle solving, etc.
Gaming is one of my main three hobbies (the other two being running and kickboxing), so I don't bother to hide it. On the other hand, I don't exactly cram it down other people's throats. Also, I'd more willingly tell people that I go running or kickboxing than gaming. Example:
On a day where I go kickboxing/running
Other Person: So what did you do today?
Me: Went to kickboxing/ went running.
On a day where I just sit inside the house all day
Other Person: So what did you do today?
Me: Nothing really (any day where I don't get a decent amount of exercise or social interaction I view as a total waste, regardless of how many S ranks I got in Devil May Cry, or how many Mega Man games I beat.)
The only real impact gaming has had on my life (aside from entertainment and the aforementioned skill improvements) is that it, combined with internet browsing/foruming, has caused me to cut down on other sedentary activities. The only thing I watch on TV nowadays is the Weather Channel, and I haven't seen a movie in theatres since Pirates of the Caribbean...........2.