Wearing any shirt that represents a video game is a dorky thing to do.
Ever see those guys who walk around with their halo shirts on? Do you ever see them and think, "Those are some awesome guys who play halo." NO, because those guys are always the absolute dweebs who take gaming way to seriously, and are bad at everything else. Personal observation of these individuals would yield the same conversations: They would talk about an awesome in game kill or match or some other shit that doesn't matter. A glance at their report cards tells you that they can't do simple math, complete a sentence, tell you what sedimentary rocks are, why december 7th 1941 is important, and that they get picked last for dodgeball.
Take another example of a friend of mine in high school. He was a big WoW nerd, and all around videogame junkie, but he never wore a WoW shirt, or any video game shirt for that matter. He spent countless hours on WoW, but he was also good at his studies (he was #3 in our class), a selection of sports that interested him, and never once talked about how cool an explosion was, or how awesome a raid on blah blah blah was.
Now granted, you might be good at other things other than WoW, but the image that wearing a videogame t-shirt conveys is that of a person is totally obsessed, that's why they want you to burn it.