Because Video Games are still a new medium and haven't matured as quickly as say, movies have since their conception.
And also because of the interactivity element, immersing yourself in a fantasy through reading or watching is one thing, but being the character who causes the murder or other bad things can be a worrying thought to some people (at least to those who think that people, and even kids can't differentiate between reality and fantasy, when they most certainly can).
Not to mention that when video games first came out, they were insanely popular amongst kids (and still are to this day, but the key demographic, or rather the one under the spotlight has become teenagers) so people tend to jump to conclusions, as I mentioned in the previous paragraph.
With teenagers, people think doubly of this because of how unpredictable they seem to the masses, and also because it is a genuine period of life that is rife with ambiguity, uncertainty in life, yourself, others etc. not to mention the emotional fluctuations caused by hormones.
What we really need if we want people to not necessarily respect games(because I know: why should we go out of our way to earn respect from people who are automatically biased against us) but to end such discussion is for games to become more human, in a way: seriousness and bleakness doesn't necessarily correlate into greatness or recognition (Max Payne 3, for eg), if a game were to come along, that not only was well written, had a great story of multiple levels with mechanics that complimented and drove the story forward, and it sold well(to be recognised by mass media), then perhaps then it would stop this nonsense.
What I am saying is we need a poster game to represent gaming: whenever people would complain that movies cause violence, people referrence Citizen Kane, or the Godfather, or when music is (supposedly)causing violence or when it cites negative messages, people bring up the good in music(or rather its genre, so for rap, someone like Akala, with rock, The Beatles, etc.).
Gaming does not have a good exemplar in the public eye, the only ones we have are Mario and Call of Duty: essentially a simple, cartoony and fun distraction that easily appeals to kids(from the layman's perspective) and a game where the story is about all "those dirty foreign countries having it out for the good ol' US of A", with a multiplayer populated mostly by petulant children with at best laissez-faire parenting.
We will get there eventually, but until we do, we might as well just stop caring when games get blamed for violence, as it is and should be meaningless white noise of hatred to us.
[small]I know I certainly stopped caring about it[/small]