The problems with the supposedly "unbiased" review

[mmiki]

This is a problem with metacritic and publisher practices though, is it really ok to try and eliminate views and scores because an outside force has decided a review aggregator should be a basis for handing out bonuses.

In my opinion, no. I wouldn't advocate anything like that.

Not to mention, that comes across as a mostly presumptive argument, as Metacritic weights scores from places like IGN heavier than scores from Polygon or other smaller sites anyway, so the chances that a game getting a low score due to a reviewer having an ideological bent having enough effect to screw a developer out of a contract bonus is astronomically low, we would have to start seeing a lot more sites than just Polygon and kotaku reviewing games that way for it to happen.

The weight given to metacritic isn't even a universal thing amongst publishers anyway, the most famous examples I can think of were New Vegas and the stipulation for Destiny, and both those games missed their target scores for completely mechanical reasons, because it takes a lot more than one or two sites knocking off points for sexualization or whatnot to throw off an entire metacritic score, and if a game was so close to the line that a single Polygon review would push it over, then the game would have to have more issues than just too much sexualization.

Well, yes, but the whole "gamers are dead" thing was published by 11 or 12 different sites. If they all agree on a certain point of view, then it becomes more than a presumptive argument. And it's not like it stops at Metacritic, we had instances where some of those sites would put pressure on developers regarding, say, sexual characters in MOBAs.

It's not just about the sexuality either, see Polygon tanking Tropico 5 score because it doesn't have the message that the reviewer is comfortable with. That review so completely missed the point that it's not even funny:

"What could have been a poignant, perhaps even hilarious commentary on the nature of narrow-minded dictators merely served to enforce the game's backward world view."

On a side note, that's the point where I stopped visiting Polygon. Because my forehead started hurting from facedesking.

My point is, it's not like those views exist in an isolated shell, they are held by people with big soapboxes. I also don't know what's the solution or if there even is one. I don't agree with trying to shut people up, in any case.

 

[OldNewNewOld]

It's impossible for a reviewer to be completely objective. Even being more objective than biased would be hard.
But there are things that the reviewer can do to make sure his bias affects the score as little as possible.

Take for example Yahtzee and Jim and Hyrule Warriors. Jim says it's a great game but if you're not a fan of Dynasty Warriors, you might find them boring. Yahtzee says the game is shit and doesn't explain why. He just goes on and on about how he dislikes because he hates the whole genre. Another thing would be Yahtzee's reviews of Monster Hunter. He didn't finish the 1st quest. Tell me, what right does he have to speak about the game? To review it? Non because he didn't experience it.

Social "justice" issues? There is rape in the game and that offends you? Review the game as if it didn't have rape and then make a disclaimer that all those who don't think can stomach virtual rape should think twice before buying it. You're shit as platforming? Don't talk about how hard the game is because it probably isn't and you're just shit at the game. And for the love of God, don't reduce points because you think it's too hard. Look at this, just look at this guy play Sonic. He reduced the final score because the game is too hard and the controls are shit. Neither is true, he's just bad at the game.

He doesn't know how to sprint, then he can't decide whether to go left or right and hits the middle. It's not the games fault.

The score should only reflect how the game runs and plays and not how it fits your political image. Sure, you can mention it. Warn your fans because they are most likely your fans and readers because they have a similar opinion like you, but make it a separate comment and not a judgment of the game. And when a game does fit your political agenda, then please separate it as well. I don't need to read how Gone Home has the best narrative in video games history. And no, I'm not exaggerating, there were several reviews that compared the shitty narrative in Gone Home with games that actually deserve the tittle of best narrative.

Don't give them the crap that I'm entitled because you obviously don't know the meaning of the word. I actually am entitled for the product if I pay for it because being entitled is actually a good thing and not a buzzword like reviewers and journalist are using the world since the ME3 disaster.

In short, if you hate genre, don't review the game form the genre. If you have a company, don't review their games. If you're incompetent in games, don't complain that they are hard. If you're stupid, don't claim that a game is smart. If you didn't play games for the last 4 generations, don't go around claiming that GameX is the best X since forever.
You can't be objective, but you can try to not be a shithead and fuck over developer and the people who like the game because you just don't like X.
Scores are really important. If a game does badly on metacritic's, the developer won't get a bonus, in some cases they might even pay a penalty. The majority of users just judge games based on the cover and the score. If they don't but it because of that and the franchise gets killed, you just fucked over a dev team and everyone who loves the franchise just because you don't know that holding B makes Sonic run faster.

I might have exaggerated the impact a single review can have, but if 10 high profile gaming sites decide that the game should get a 9/10 because the MC is a white dude or because there are stripper that you can kill, it can and will have negative consequences.

EDIT: Another thing that annoys me in reviews is that every times someone else brings up objectivity, the review gets defended with "you can't objectively judge games". Okay, I'm fine with that. But why does the reviewer try to impose his opinion as facts? Why does he/she say X is bad or good rather than I dislike X? X is boring rather than I don't find it interesting? Zelda is always the same game over and over again rather than I didn't play the game but I want to talk shit about it because I know my fans will watch it and defend me despite knowing that I didn't play the game?

 

[Scootinfroodie]

I cannot believe this goes three pages. There is no such thing as an "unbiased" or objective review. The very concept of it is absurd, and has been spoofed effectively in the past.

It depends on what you mean by unbiased and objective. One thing I've consistently noticed is the confusion of journalistic and philosophical objectivity. When you boil it all down, basically nothing is philosophically objective without allowing certain assumptions. The topic is basically designed to devolve into philosophical bickering and the whole "how do you know you know something" line of thought. However, journalistic objectivity is absolutely worth discussing, even within the context of a review. Examples of journalistic objectivity being pursued taken in the past few months?
For starters, the willingness by a number of games media sites, including this one, to attempt to financially separate writers from subjects
The criticism of the actions suggested by the GameJournoPros list is also a good example.
And finally the Escapist's willingness to allow discussion and interview both sides of an online conflict that has become needlessly hostile

In terms of reviews specifically, this has been discussed already
There ought to be disclosure of bias, including any and all ties or benefits rendered to the reviewer by the game's creator/PR agent/publisher/etc. In terms of the review and its goals, as well as the efficacy of the writer themselves, there's a number of different thought processes that are all perfectly up for discussion

Reviews will exist somewhere along a scale of relatively impartial to highly personalized. The former is likely to be relevant to a large number of people whilst at the same time not being particularly useful...of necessity it will be highly generic. The latter will be relevant to a small number of people, but for those to whom it is relevant it will be highly useful, as it will reflect their tastes.

Which is why you can write more than one piece on a subject. They don't suddenly fade from existence once the first article is written. I seem to recall multiple articles talking about different aspects of Mass Effect 3 existing in tandem on the same website (and sometimes even by the same author).

All reviews, no matter how partial or biased, have a right to exist.

Nobody's saying they shouldn't exist at all. What they're saying is that websites that want people to consider them respectable/worth reading should hold themselves to, at the very least, some pretty basic standards. You have to remember that these complaints are being leveled at mainstream sites, not specialists. Not only is their knowledge of the subject matter (whether the subject is the game/genre itself or any and all shoehorned in philosophical and political ramblings) generally fairly surface level (if it's not entirely misguided), but they are writing pieces that *are* directed at a large group of people

If a review aggregate site like Metacritic is including "biased" reviews in their score, as they clearly do, and that effects bonuses or what not, as has been the case a couple of times, the responsibility for that lies on the publisher who tied financial recompense to something as ephemeral as a metacritic score. If you want to protest, complain to the publisher. If you want to get your activist hat on, boycott their games and let them know why.

The argument was that reviews don't affect the developer. I simply pointed out how that isn't true.

Don't seek to REMOVE THE REVIEWS.

Holding something to standards != removing it. There are basically no standards whatsoever for most of the mainstream gaming sites. I find it absurd that you see no issue from this even from the perspective of holding the attention of one's core market, as this lack of quality control is one of the driving forces behind vitriol levelled against the sites in question

 

[Fishyash]

In regards to bias in reviews, if there is a personal bias, let's say by genre as an example, I would much rather read a review with a bias towards the genre than against it, because if someone is biased towards a genre they are much likely going to spout an informed opinion.

With that said though, I don't give a shit about bias, if I read a fighting game review where the critic was smothering the game in praise, and it turns out he/she never played a fighting game before, I deem it a low quality review (although I would hope his/her passion in fighting games would stay, and let's say 5 years later he/she would be able to review another fighting game with a much more informed opinion).

With that said, financial or relationship biases are no good for reviews and if you are in that much need of money then you should put your site behind a paywall or something.

But let's get creative with an example of personal, rather than financial or relationship based, bias:

If I play a game where all the mechanics are excellent, production values are good, it's well optimized and the story holds up well, but I hate one aspect of the game so much it sours the whole experience (making me strongly dislike the game). How then should I then rate it?

Bolded part is the rating. "Strongly dislike the game" which would be a 1/5. I don't get what's so difficult, confusing or bad about review scores to be honest.

 

[BloatedGuppy]

It depends on what you mean by unbiased and objective.

I mean "unbiased" and "objective". Your mileage may vary.

One thing I've consistently noticed is the confusion of journalistic and philosophical objectivity.

I'm not sure why people keep trying to apply standards of journalistic objectivity to media criticism. I read reviews to get an individuals evaluation of a game. I want to know their personal take on it. I don't want them dryly rattling off a feature list.

And fuck, even journalists can't even agree on what "journalistic objectivity" is supposed to mean when it pertains to actual journalism: http://www.cjr.org/feature/rethinking_objectivity.php?page=all

There ought to be disclosure of bias, including any and all ties or benefits rendered to the reviewer by the game's creator/PR agent/publisher/etc. In terms of the review and its goals, as well as the efficacy of the writer themselves, there's a number of different thought processes that are all perfectly up for discussion

Disclosure of bias in terms of "The publisher totally paid for a good review"? Absolutely. Disclosure of bias as in "I attended liberal arts college and tend to vote Democrat and align myself with these social causes and enjoyed this book by this controversial author..." so that the zealots demanding any and all political leaning (that they disagree with) be smote from the text? I do not concur.

Which is why you can write more than one piece on a subject. They don't suddenly fade from existence once the first article is written. I seem to recall multiple articles talking about different aspects of Mass Effect 3 existing in tandem on the same website (and sometimes even by the same author).

Yeah, you could do that. Or you could just write one piece. It doesn't really matter either way.

Nobody's saying they shouldn't exist at all.

Actually a lot of people are, in fact, saying exactly that.

What they're saying is that websites that want people to consider them respectable/worth reading should hold themselves to, at the very least, some pretty basic standards. You have to remember that these complaints are being leveled at mainstream sites, not specialists. Not only is their knowledge of the subject matter (whether the subject is the game/genre itself or any and all shoehorned in philosophical and political ramblings) generally fairly surface level (if it's not entirely misguided), but they are writing pieces that *are* directed at a large group of people.

Go to Alexa and check out site traffic on click-bait sites like Kotaku or Polygon. Tell me that they're not aware of how their business works and how get people to "consider them worth reading". They clearly have a handle on what they're doing. Whether you or I personally approve of them is probably irrelevant to their business model.

Holding something to standards != removing it.

Again, you hold something to your personal standard by voting with your wallet...or in this case your mouse I guess. Don't like it? Don't read it. I don't see what the purpose of the black list or the mailing campaign was if not to silence dissenting voices. Heck, I even had one guy tell me "majority rules" should determine who gets to speak and who doesn't.

There are basically no standards whatsoever for most of the mainstream gaming sites. I find it absurd that you see no issue from this even from the perspective of holding the attention of one's core market, as this lack of quality control is one of the driving forces behind vitriol levelled against the sites in question

Who said I "see no issue" with it? I've been quite up front with what my specific issues with the "gaming press" are. "Biased reviews" just happens to not be one of them.

 

[EternallyBored]

I think you and I agree a lot more than we may both think. What are your opinion's on Polygon's Bayonetta 2 review?

After having just read it, it's a review that covers a lot of the gameplay points, and misses others, given that its a WiiU game, I can understand why graphics weren't a major consideration, the bulk of the review seemed to swing from talking about gameplay to the problems the reviewer had with the sexualization. The review also doesn't talk about the story as much as I usually like in my reviews.

The sexualization complaints are ones I both agree with the author on some points and disagree on others, and I can understand why it would hurt his enjoyment of the game, the text praises much of the game whilst deriding the presentation of bayonetta herself, and that while he enjoyed the action, the propensity for the game to have heavily sexualized moments interspersed during the fights seriously impacted his enjoyment of the game.

The podcast at the end features the reviewer talking about how he really liked the mechanics, but that there was a subjective component that seriously caused him issues, and he justifies his views fairly well even if I disagree with the degree with which they may impact. He also talks about how a couple of other negatives for the game were not mentioned for space reasons and how the scoring system works.

The score seems unusually low, but that's not so much Bayonetta specifically as it seems to be an issue with polygon in general, or at least a few reviewers on it. Polygon is sort of the anti-IGN, where IGN takes flak for criticizing a game heavily in the written review but still giving it an 8.5-9.5. Polygon seems to have a lot of reviews that heavily praise a game whilst only listing a few negatives, or negatives that don't seem like a big deal, and then sticking it with a 7-8 range score. Even games that don't list ideological complaints will often get very low end scores of 7-8.5 on Polygon, so that seems more like just how their scoring rubric breaks down. As a side note, the person who writes the text of the review isn't the sole arbiter of the score review, Polygon uses some kind of weird median score system amongst multiple staff members.

Overall, it's not my cup of tea, but I can easily see how such a review would be useful to some people, especially if taken in consideration with a range of other reviews and viewpoints on the game, from the other people giving it a 7-8 to the people giving it perfect scores.

 

[DrOswald]

I keep seeing, and hearing, people talk about the call for unbiased reviews. But really, there can be no such thing from any human being that has ever played a game or has any opinions on anything. At least if you want an honest review.

Now, for sure, I don't think you have any business reviewing a game made by a friend or someone who you have certain financial ties to (such as where you stand to gain financially from the success of the game) or when you have worked on creating the game. That kind of bias through relationship is easy to identify, and avoid, and is generally considered as being corrupt - rather than just biased.

But let's get creative with an example of personal, rather than financial or relationship based, bias:

If I play a game where all the mechanics are excellent, production values are good, it's well optimized and the story holds up well, but I hate one aspect of the game so much it sours the whole experience (making me strongly dislike the game). How then should I then rate it?

Should I disregard my experience and judge the game wholly on its mechanics and execution, or should I take my experience into account and rate it based on how I experienced it - meaning based on my own values and enjoyment?

I think cases like these illustrate the problems with assigning a numbered score; as scores are problematic because it's often all people look at, rather than the actual review. Without scores the actual review does, in my opinion, carry more weight.

But as it is, ultimately, it's the number assigned at the end that carries actual weight.[footnote]I do not review games, but if I did I would not assign them scores. I'd possibly assign them arbitrary things like: "I give this game 4 penguins wearing funny hats, 25 sad seals and one seal whisperer - there to help cheer up the sad seals and teach them how to love again." or "One potted plant and a companion pillow with the print of your favorite banana, half peeled."[/footnote][footnote]Captcha: that will not work. Shut up captcha, I'll give whatever random nonsense that pop up in my head at the time as a hypothetical scores if I want to.[/footnote]

Can art be reviewed without bias? Well, no. A critic will always be influenced by their own prejudices and preconceived notions. But that does not mean a reviewer should embrace that bias and let it become the driving factor of the review. And it certainly does not mean that the quality of art is entirely subjective.

A good reviewer will acknowledge these factors and attempt to correct for them in their review. Not by changing their opinion for the final verdict but by accounting for them. Jim Sterling's recent review of Hyrule Warriors is an excellent example of this. He fully acknowledges that the reasons he likes the game may not apply to everyone and effectively communicates that idea and those factors that lead to his near perfect review of the game.

Another good example is Polygon's Bayonetta 2 review. He has nothing but praise for the mechanics of the game, but dislikes the sexualization of the game so much that it, in his opinion, taints the entire game. Fair enough, he says what he really thinks and I know to completely ignore his opinion of the game.

A "biased" review will fail to make these corrections.

Furthermore, while we can't truly say there is such a thing as the unbiased review, we can say for sure there is such a thing as the overly biased review - the review that misrepresents or twists the truth or just outright lies in a review for the sake of pushing an agenda, be that to appease a big publisher, push a specific political agenda, or to help out a friend. We know these types of reviews happen.

The fact that no review is capable of being truly unbiased is not a licence to embrace your own bias or strait up lie.

 

[Knight Captain Kerr]

Objectivity is both impossible and undesirable. People have already posted links to the quite humorous objective review Jim Sterling did of Final Fantasy 13 so I'm not going to.

No piece of art is objectively good or bad, it's all subjective. Art is not a term that denotes quality, just because something is art doesn't mean it's good. Reviews are by their nature subjective opinion pieces. Also people talking about just judging mechanics objectively, that's subjective too. For example I don't like axis inversion, some people do. Even if you think the controls are poor sometimes that's the entire point like in Surgeon Simulator, it's fine to still not like it though.

My advice? Read/listen to/watch a number of reviews before you decide if you want to get a game or not. Find reviewers you trust and like even if that's just asking your friend what they think of it or asking on a forum if "X" is worth playing. You don't need to rely on "professional" reviews. You can also respond to someone's review if you disagree with it and stop reading someone's reviews if you think they're crap.

 

[KazeAizen]

Honestly, completely unbiased reviews sound about as enjoyable as watching paint dry. If a reviewer is allowed to show his excitement, and his passion toward something, then it can lead to far more creativity and will be more enjoyable to watch/read as a result. Force someone to give a boring, unbiased and "professional" review on something, and keep certain emotions locked up, would make for something even more disingenuous, in my opinion.

I'd much rather reviewers speak openly and honestly.

Seriously. We all have our own hang ups and fixations. I think even Moviebob addressed this issue in one of his Big Pictures a few months ago. Besides I don't inherently see "bias" as a totally objectionable thing either. I mean if you are on the opposing side of the opinion it could potentially open up your mind to thinking differently, or if nothing else you learn that this reviewers tastes don't line up with yours so you just find someone who is maybe similar so you can gauge if you truly want to watch that movie or play that game.

Heck I think one of Moviebob's "bias" when it comes to comic book movies is that he is sick to death of the grim and gritty aesthetic being applied to a lot of them when he thinks they should be a lot more light hearted and fun. Even if he likes the first two Nolan Batman movies. Heck its part of the reason I see where he's coming from on Man of Steel but I still disagree with him and think Man of Steel is a much better movie than he gives it credit for.

On the other hand Jim Sterling loves him some Dynasty Warriors and Alien stuff. So you bet your ass that even though I know he's biased towards those things I'm reasonably certain I can trust his opinion on anything related to it. Which is why I was happy with his Hyrule Warriors review. Fans are fans after all and that's really what we all are to begin with, professionally paid or not.

 

[Scootinfroodie]

I mean "unbiased" and "objective". Your mileage may vary.

I know that when asked for clarification for the sake of honest discussion *I* always find restating terms that mean different things depending on context helps

I'm not sure why people keep trying to apply standards of journalistic objectivity to media criticism. I read reviews to get an individuals evaluation of a game. I want to know their personal take on it. I don't want them dryly rattling off a feature list.

And fuck, even journalists can't even agree on what "journalistic objectivity" is supposed to mean when it pertains to actual journalism: http://www.cjr.org/feature/rethinking_objectivity.php?page=all

I do believe there's a middle ground between feature list and spending most/all of a review not talking about gameplay or utilizing a popular release to soapbox and then give a rating out of 10. There's a point in general coverage and writing between a dry detached press release and calling the creator of a game a 14 year old that most professionals should be able to navigate.

As far as the ongoing debate over the standards of journalism, that's perfectly welcome. Blatant dismissal of the idea, however, shows a lack of respect for the reader's time and patronage.

Disclosure of bias in terms of "The publisher totally paid for a good review"? Absolutely. Disclosure of bias as in "I attended liberal arts college and tend to vote Democrat and align myself with these social causes and enjoyed this book by this controversial author..." so that the zealots demanding any and all political leaning (that they disagree with) be smote from the text? I do not concur.

Disclosure as in dinners, flights, "swag", outright bribes, previous employment at the company in question, living with or being in a relationship with a developer, funding the developer on a regular basis etc.

As for the second portion, while it's valid to come at a critique from a particular philosophical or political viewpoint, I don't see why the consumer should accept soapboxing in the middle of a product review. This is less about objectivity and more about quality and professionalism however.

Yeah, you could do that. Or you could just write one piece. It doesn't really matter either way.

I don't know if you've noticed, but reviews tend to be a certain length so that people read them. If you try to cram a lengthy post-modernist critique in the middle, something gets cut
Sounding like a broken record, but again, look at the RPS Skullgirls review

Actually a lot of people are, in fact, saying exactly that.

Most of what I've seen is a request that they remain separate. Perhaps from your position of insistence that the status quo remain, their challenge is read as an attempt at removal. I have a hard time believing that "a lot" of people are dumb enough to believe that you can keep things "off teh interwebs"

Go to Alexa and check out site traffic on click-bait sites like Kotaku or Polygon. Tell me that they're not aware of how their business works and how get people to "consider them worth reading". They clearly have a handle on what they're doing. Whether you or I personally approve of them is probably irrelevant to their business model.

Alexa rankings, from what I've read recently, are a bit sketchy. I'm not entirely sure why everyone is so reliant on them

It IS worth noting, however, that investors have pulled support, and that RPS and Gamasutra seem to be having a bit of trouble, so there's definitely more than a few signs that, in fact, the contrary is true. I don't suspect any Gawker site will close or change tactics any time soon, but that has more to do with what the site is and how it works. I wouldn't be using it as a positive example though, no matter what political viewpoint they espouse

Also on the flip side, confirmed 10% increase in Escapist traffic after they allowed discussion and changed their reporting methodology

Again, you hold something to your personal standard by voting with your wallet...or in this case your mouse I guess. Don't like it? Don't read it. I don't see what the purpose of the black list or the mailing campaign was if not to silence dissenting voices. Heck, I even had one guy tell me "majority rules" should determine who gets to speak and who doesn't.

And people have done so. This thread was clearly made with some form of discussion in mind, however, so if your goal was to point out what people are already doing, and attempt to dismiss or outright stop discussion, why did you post?

Who said I "see no issue" with it? I've been quite up front with what my specific issues with the "gaming press" are. "Biased reviews" just happens to not be one of them.

Where have you done this exactly? Without further explanation, your posts in this thread only act in defence of the status quo. As far as "biased reviews" go, you clearly agree that aspects of writer/subject involvement are worth knowing and dealing with. Other aspects of acceptable reviewing can and should be discussed and that, once again, ought to be the purpose of this thread

 

[BloatedGuppy]

I know that when asked for clarification for the sake of honest discussion *I* always find restating terms that mean different things depending on context helps.

Fair enough. Objectivity - completely fact based. No subjective opinion allowed to enter whatsoever.

Unbiased - Reviewer is not permitted to allow any personal beliefs or outlooks affect the content of their prose. It must be completely neutral.

Both those things are argued for by individuals in this thread. Notably, they seem particularly incensed at the idea of beliefs they do not share being espoused, or a game receiving a score they do not agree with.

I do believe there's a middle ground between feature list and spending most/all of a review not talking about gameplay or utilizing a popular release to soapbox and then give a rating out of 10. There's a point in general coverage and writing between a dry detached press release and calling the creator of a game a 14 year old that most professionals should be able to navigate.

What amount of subjectivity is too much? Is there a metric we can use to determine this? How do we determine lack of professionalism? Is there a metric for THAT? Seems like it would be down to the reader's opinion to me.

Disclosure as in dinners, flights, "swag", outright bribes, previous employment at the company in question, living with or being in a relationship with a developer, funding the developer on a regular basis etc.

Sure, to all of that.

As for the second portion, while it's valid to come at a critique from a particular philosophical or political viewpoint, I don't see why the consumer should accept soapboxing in the middle of a product review. This is less about objectivity and more about quality and professionalism however.

I don't really view it as my lookout what other consumers want or don't want in a review.

I don't know if you've noticed, but reviews tend to be a certain length so that people read them. If you try to cram a lengthy post-modernist critique in the middle, something gets cut
Sounding like a broken record, but again, look at the RPS Skullgirls review

I've read reviews of many lengths and varieties. Whether I read them or not is usually contingent on my interest in the subject matter and/or appreciation of the writer.

Most of what I've seen is a request that they remain separate. Perhaps from your position of insistence that the status quo remain, their challenge is read as an attempt at removal. I have a hard time believing that "a lot" of people are dumb enough to believe that you can keep things "off teh interwebs"

My position of what now? Sounds rather like you're attempting to assign a position to me based on my disagreement with you.

I have a hard time believing it too, but there you have it.

Alexa rankings, from what I've read recently, are a bit sketchy. I'm not entirely sure why everyone is so reliant on them

Lack of other available data?

It IS worth noting, however, that investors have pulled support, and that RPS and Gamasutra seem to be having a bit of trouble, so there's definitely more than a few signs that, in fact, the contrary is true. I don't suspect any Gawker site will close or change tactics any time soon, but that has more to do with what the site is and how it works. I wouldn't be using it as a positive example though, no matter what political viewpoint they espouse

I'm aware of the slumps in RPS and Gamasutra, and have commented on them before. You'll note that, particularly in the former case, the slump in question was mirrored at this time last year, and views are climbing again. Wouldn't it be natural to assume this is a cyclical readership issue driven by a lack of strong PC releases?

Also on the flip side, confirmed 10% increase in Escapist traffic after they allowed discussion and changed their reporting methodology

What are previous rises in traffic attributed to?

And people have done so. This thread was clearly made with some form of discussion in mind, however, so if your goal was to point out what people are already doing, and attempt to dismiss or outright stop discussion, why did you post?

The thread poses a question, I responded to that question. Am I "outright stopping discussion"? Oh god...am I censoring people? Is that what's happened?

Where have you done this exactly? Without further explanation, your posts in this thread only act in defence of the status quo. As far as "biased reviews" go, you clearly agree that aspects of writer/subject involvement are worth knowing and dealing with. Other aspects of acceptable reviewing can and should be discussed and that, once again, ought to be the purpose of this thread

No, they really don't. They act as refutation of a particular idea, that of the "objective" review. However, in this forum's current entirely polarized "you are with us or against us" atmosphere, I am not surprised the interpretation was that I delight in all aspects of gaming journalism because I defended one.

 

[Scootinfroodie]

Fair enough. Objectivity - completely fact based. No subjective opinion allowed to enter whatsoever.

Unbiased - Reviewer is not permitted to allow any personal beliefs or outlooks affect the content of their prose. It must be completely neutral.

Alright, so you're using them within a different context. Subjectivity is, on some level, going to bleed into everything you do. You've decided upon a context where there is utterly no win condition for the opposition even outside of talking about games journalism.

Both those things are argued for by individuals in this thread. Notably, they seem particularly incensed at the idea of beliefs they do not share being espoused, or a game receiving a score they do not agree with.

While I'm not going to doubt the entire existence if these ideas, I doubt their prevalence in this particular thread. I feel that this may be, intentionally or otherwise, an instance of strawmanning

What amount of subjectivity is too much? Is there a metric we can use to determine this? How do we determine lack of professionalism? Is there a metric for THAT? Seems like it would be down to the reader's opinion to me.

It most certainly is down to opinion. This thread is about discussing that opinion. How much do you feel is too much?

I've read reviews of many lengths and varieties. Whether I read them or not is usually contingent on my interest in the subject matter and/or appreciation of the writer.

Sure, but not everyone has time to do so. As a result you're going to get a large number of people reading a small number of trusted sites. If those sites betray that trust, people complain. As you've said before, ultimately the solution is to leave, but simply leaving does not help the writer in question improve or avoid losing other readers.
In my case, other people had already made the complains I would have of RPS' methods, and I haven't really visited that site since

My position of what now? Sounds rather like you're attempting to assign a position to me based on my disagreement with you.

I'm commenting on your implied position that this is a non-issue, and can further draw from my now-confirmed suspicions as to your definitions of the words involved.

I have a hard time believing it too, but there you have it.

What I'm saying is, perhaps it's a result of your perception of the overall conversation, and not necessarily their actual position

Lack of other available data?

Lack of other data doesn't make potentially biased data more accurate though. Other sites not reporting on Gawker rumors doesn't make Gawker any less sketchy, for instance

I'm aware of the slumps in RPS and Gamasutra, and have commented on them before. You'll note that, particularly in the former case, the slump in question was mirrored at this time last year, and views are climbing again. Wouldn't it be natural to assume this is a cyclical readership issue driven by a lack of strong PC releases?

If it is, it certainly isn't cutting it anymore for the RPS staff, as they've introduced a new monetization system to their site

What are previous rises in traffic attributed to?

I'd have to know the context of the previous traffic fluctuations and whether or not they actually occurred. Again, Alexa has been accused of inaccurate reporting methods and even fudging numbers

The thread poses a question, I responded to that question. Am I "outright stopping discussion"? Oh god...am I censoring people? Is that what's happened?

The thread posed a question, and you dismissed it. I don't see the purpose in condescending lecturing and dismissal of discourse, but perhaps you do. More power to you I suppose

No, they really don't. They act as refutation of a particular idea, that of the "objective" review. However, in this forum's current entirely polarized "you are with us or against us" atmosphere, I am not surprised the interpretation was that I delight in all aspects of gaming journalism because I defended one.

Dismissing a conversation about current issues is, implicitly or explicitly, defence of the way things currently stand. Whether this is in relation to all of games "journalism" or simply the topic of reviews depends on the conversation had. It's interesting that you accuse me of attempting to attribute positions when that's all you've done thus far in this thread, aside from insisting that the conversation is "absurd" and that you "cannot believe it lasted 3 pages"

 

[BloatedGuppy]

Alright, so you're using them within a different context. Subjectivity is, on some level, going to bleed into everything you do. You've decided upon a context where there is utterly no win condition for the opposition even outside of talking about games journalism.

I am using them within a common context, supported by the definitions of the words and employed by others in the thread.

While I'm not going to doubt the entire existence if these ideas, I doubt their prevalence in this particular thread. I feel that this may be, intentionally or otherwise, an instance of strawmanning

Well that is certainly your prerogative. People are often given to doubt the existence of inconvenient reality.

It most certainly is down to opinion. This thread is about discussing that opinion. How much do you feel is too much?

Personally? Or in general?

What I'm saying is, perhaps it's a result of your perception of the overall conversation, and not necessarily their actual position

There is always room for miscommunication in a written medium. That said, some people have been pretty clear about their belief that 'biased reviews' should not exist, that journalists who engage in them should be fired, that "media corruption" equates to "having an ideology", etc, etc, etc.

Lack of other data doesn't make potentially biased data more accurate though.

That's fine. If you want to disregard it, disregard it. What data do you use to support the assertion that they are losing customers?

If it is, it certainly isn't cutting it anymore for the RPS staff, as they've introduced a new monetization system to their site

Yep, I know what's going on with RPS. Do sites only engage in monetization systems during periods of dramatic view loss? We can speculate, but we have nothing to base it on. RPS has also been declining in overall quality for many months now, having lost some of their better staff writers. Could that also attribute to view loss? Or must we ascribe it Gamer Gate because that fits a narrative?

I'd have to know the context of the previous traffic fluctuations and whether or not they actually occurred. Again, Alexa has been accused of inaccurate reporting methods and even fudging numbers.

Oh god, are they part of the social justice liberal media conspiracy too? I can't keep track any more. What motivation are they meant to have for fudging numbers?

The thread posed a question, and you dismissed it. I don't see the purpose in condescending lecturing and dismissal of discourse, but perhaps you do. More power to you I suppose.

Personally *I* don't see the point in characterizing someone's tone as "condescending lecturing" whilst masquerading as interested in collegial debate. Your mileage may vary.

Dismissing a conversation about current issues is, implicitly or explicitly, defence of the way things currently stand. Whether this is in relation to all of games "journalism" or simply the topic of reviews depends on the conversation had. It's interesting that you accuse me of attempting to attribute positions when that's all you've done thus far in this thread, aside from insisting that the conversation is "absurd" and that you "cannot believe it lasted 3 pages"

Yep. That's "all I've done", and the entire summation of my argument can be boiled down to two statements. Good faith discussion. Really enjoying it.

 

[Kerethos]

What I would be interested to know is: having read our responses, how you yourself would now answer the central question in your first post:

If I play a game where all the mechanics are excellent, production values are good, it's well optimized and the story holds up well, but I hate one aspect of the game so much it sours the whole experience (making me strongly dislike the game). How then should I then rate it?

Do you slam the game, do you merely subtract a point or just scrap the entire review?

I'd try to talk about what's good and then go into full lengths to explain why I still, despite the good parts, dislike the game.

And rather than just make the final rating based on some sort of "+points for good -points for bad" -formula I'd try to give a number representing my overall enjoyment of the game (provided I had to give a number, which I'd rather not).

So if I found the game to be a technical masterpiece, but horribly offensive, I'd rate it something like a 1/10 - "No amount of excellent mechanics can make me able to enjoy something I find this offensive."

In regards to bias in reviews, if there is a personal bias, let's say by genre as an example, I would much rather read a review with a bias towards the genre than against it, because if someone is biased towards a genre they are much likely going to spout an informed opinion.

I strongly object to this attitude of "you should only review games of genres you like and have extensive experience with".

If you normally don't enjoy platformers, and people who normally never enjoy platformers never review them, how would you ever be encouraged to broaden your horizons when someone with similar tastes as yours finds a game they actually like? I don't normally enjoy racing games, but I've still encountered some that where really fun.

By your review requirements a reviewer should only play games of genres they like, and games with broader appeal should never be reviewed by anyone who's not already likely to be a fan.

I mean if Yahtzee played a JRPG even he had to admit was fun I would be much more likely to check out. Even though I had my fill of JRPG's after FFX (which I found, for the most part, quite fun).

 

[grassgremlin]

What kind of review would this be then?
https://www.youtube.com/watch?v=8S6myWz3Kzg

 

[Scootinfroodie]

I am using them within a common context, supported by the definitions of the words and employed by others in the thread.

They are one common context, and not necessarily the one being used by the people you're arguing against.

Well that is certainly your prerogative. People are often given to doubt the existence of inconvenient reality.

Condescension is a great replacement for arguments and evidence

Personally? Or in general?

Either, or both

There is always room for miscommunication in a written medium. That said, some people have been pretty clear about their belief that 'biased reviews' should not exist, that journalists who engage in them should be fired, that "media corruption" equates to "having an ideology", etc, etc, etc.

Again, it depends on what they're considering "biased". People who write about their employers, SO's etc. and don't disclose should be suspended or terminated for sure. As for the last bit, that's most certainly not what I'm seeing. Could you quote the specific bit of text where somebody considers having a political or philosophical affiliation to be worth firing someone over?

That's fine. If you want to disregard it, disregard it. What data do you use to support the assertion that they are losing customers?

Losing customers in general? Word of mouth? My own experiences?
Larger moving of readers? Maybe the pulling of ads?

Yep, I know what's going on with RPS. Do sites only engage in monetization systems during periods of dramatic view loss? We can speculate, but we have nothing to base it on. RPS has also been declining in overall quality for many months now, having lost some of their better staff writers. Could that also attribute to view loss? Or must we ascribe it Gamer Gate because that fits a narrative?

When did I mention a twitter hashtag, or attribute customer loss to that? These issues have been around for years

As for RPS specifically, if we look at similar sites/business models, a sudden shift in model is a result of the previous system not working. While it's possible that RPS is just gettin' greedy, it's not really probable.

Oh god, are they part of the social justice liberal media conspiracy too? I can't keep track any more. What motivation are they meant to have for fudging numbers?

Where did I mention a conspiracy? For someone who complained about attributed positions, you seem to be doing nothing but. A watchdog group accused the site of bumping up the numbers of left wing sites, and lowering those of right wing sites when those sites were reporting growth. Whether or not that's true, there are countless articles on how Alexa's ranking system is flawed either in the sense of it being inaccurate (generally due to sampling methods) or easily gamed (generally for the same reasons)

Personally *I* don't see the point in characterizing someone's tone as "condescending lecturing" whilst masquerading as interested in collegial debate. Your mileage may vary.

When one is being condescending and is showing no interest in actual discussion, what is the appropriate term? I'm perfectly interested in discussion. You're assertion that I'm "masquerading" holds no weight

Yep. That's "all I've done", and the entire summation of my argument can be boiled down to two statements. Good faith discussion. Really enjoying it.

That's some killer reading comprehension. If your entire position can be boiled down to two statements, how can it also be a third thing (attribution of positions). Much of your "argument" has been that the argument is stupid, the rest has boiled down to:

Insisting that you, I, or someone else has stated X without actually explaining where or how that happened
Insisting that everyone is using the context of two words that you are using, by virtue of you stating that it's the case
Making snarky jabs instead of actually holding any kind of position that would facilitate discussion

Please explain what portion of your position I'm expected to honestly discuss. Feel free to use points from other threads, but only if you're going to bother actually listing them here and not vaguely reference them like I'm expected to read your entire posting history throughout however many months count for this exercise

 

[Erttheking]

You know what I've noticed? A lot of people in this thread don't really have a problem with reviews that have bias. They have a problem with reviews that have NEGATIVE bias. They say that they only want genres to be reviewed by people familiar with it. That's bias towards the subject matter, but they don't have a problem with that.

 

[BloatedGuppy]

Condescension is a great replacement for arguments and evidence

This is the second charge of "condescension" from you, and I was made curious. I read back through your posts in this thread, and then your post history in general. I don't know if you're oblivious to your own posting style, or simply calling out behavior your recognize in yourself. Yes, I am being condescending. I am most definitely not alone in that. Alas, I can't control your behavior, I can only control my own, so I'll try to be less of an ass. I would appreciate it if you would return the favor.

Either, or both.

For myself, "too much" would be that point at which the bias distracted from my enjoyment of the review. Either by completely obfuscating the information I'd come to find, or by presenting an ideological perspective I found unpalatable.

In general, not at all. It's not for me to police the internet and decide what people should or shouldn't be reading.

Again, it depends on what they're considering "biased". People who write about their employers, SO's etc. and don't disclose should be suspended or terminated for sure. As for the last bit, that's most certainly not what I'm seeing. Could you quote the specific bit of text where somebody considers having a political or philosophical affiliation to be worth firing someone over?

Okay, this is one of several points that provoked my "bad faith" comment. I know from your posting history that you have spent time in the Gamer Gate threads, including the mega thread. From your tone and the people you choose to debate, I'm relatively certain I know which side of the debate you land on. I know you're aware of the existence of a media blacklist, and I know you're aware there have been calls to drum people out of the industry. I know you're one of the major reasons for that call is a presumed "ideological bias" in the media, and that the ideology in question is "liberal". I know you're aware that there have been calls to have people fired, to crumble websites, to "go to war".

I'm not going to data mine 1600 pages of thread to provide you quotes. I'm sure you're a lovely guy, but my investment in proving this point to you does not run that deep. You can dismiss it all you want on grounds of insufficient evidence, if that is your preference. Burden of proof is on me, and I'm unwilling to provide it, because it's a lot of fucking effort for something that ultimately doesn't matter very much to either of us. If you wish to characterize me as a fabulist, you may do so. I know what I read. There is a reason for my rancor on this issue.

I don't require quote mining to know that people who oppose Gamer Gate have said or done terrible things. It stands to reason. It's a polarized, aggravated debate (that for some reason has become deeply politicized). I know that people on both sides are saying ridiculous things the same way I know the water in this glass is wet.

There's a reviewer named Tom Chick. He's been plying his trade in this industry for a long time. He's known for a couple of things...idiosyncratic taste in games, and a propensity for being honest whilst employing his entire 5 point scale. He often gives popular games he didn't enjoy bad scores, most famously Deus Ex. He is LOATHED for it. On this website, a year or two ago, there was a thread discussing his review of some Halo game or other. He was called a troll and a malingerer, a clickbaiter and an attention whore. Some people don't like "honest". Some people like confirmation of their existing beliefs. Quite frequently when someone charges a review with being "too biased", what they mean is "too not what I believe".

Losing customers in general? Word of mouth? My own experiences? Larger moving of readers? Maybe the pulling of ads?

That's fine, as long as you acknowledge this isn't any more reliable than Alexa. If I offered up "my own experiences" or "word of mouth" to contradict your beliefs on this subject I highly doubt you'd put any stock in them.

When did I mention a twitter hashtag, or attribute customer loss to that? These issues have been around for years

I'm referring to Gamer Gate in general. The commonly accepted narrative is that Gamer Gate "won", and that the sites who wronged them were devastated and bleeding customers. I've never seen any concrete evidence to support that.

As for RPS specifically, if we look at similar sites/business models, a sudden shift in model is a result of the previous system not working. While it's possible that RPS is just gettin' greedy, it's not really probable.

Honestly, neither one of us knows anything. We're both just speculating.

Where did I mention a conspiracy? For someone who complained about attributed positions, you seem to be doing nothing but.

Yes, I attributed a sympathetic if not wholly involved stance in Gamer Gate to you. That was unfair of me.

A watchdog group accused the site of bumping up the numbers of left wing sites, and lowering those of right wing sites when those sites were reporting growth.

A watchdog group. Which watchdog group? Could it have been a...right wing watch dog group? I'm not even trying to be snarky, it just...stands to reason.

Whether or not that's true, there are countless articles on how Alexa's ranking system is flawed either in the sense of it being inaccurate (generally due to sampling methods) or easily gamed (generally for the same reasons)

Yes, a quick Google search confirms this to be true. I'm certainly not married to Alexa, the first time I even used it was when I Googled something along the lines of "site traffic" when wanting to see if sites really were taking it on the nose. If there a better tool, I'll happily use the better tool.

Please explain what portion of your position I'm expected to honestly discuss.

My OP, despite hanging on a particular interpretation of the terms...one employed almost universally through the thread up to that point...was not unreasonable. Yes, I exhibited astonishment that there was even debate. I also clarified my opinion and provided context. Your response to this was to wade in swinging and say I was being "absurd". I was already in a bad mood due to the poor judgment I showed earlier in the day reading the contents of a variety of Gamer Gate threads on the Off-Topic Forum, so I was not in the mood to be snotted at about semantics. Things devolved from there.

The unfortunate thing is I've actually tried to stay relatively moderate in this debate (not THIS debate, but the GG debate), and clearly I'm becoming a bit polarized myself if I'm reacting THAT aggressively to perceived agenda. I need to take a break from reading this shit.

 

[veloper]

What I would be interested to know is: having read our responses, how you yourself would now answer the central question in your first post:

If I play a game where all the mechanics are excellent, production values are good, it's well optimized and the story holds up well, but I hate one aspect of the game so much it sours the whole experience (making me strongly dislike the game). How then should I then rate it?

Do you slam the game, do you merely subtract a point or just scrap the entire review?

I'd try to talk about what's good and then go into full lengths to explain why I still, despite the good parts, dislike the game.

And rather than just make the final rating based on some sort of "+points for good -points for bad" -formula I'd try to give a number representing my overall enjoyment of the game (provided I had to give a number, which I'd rather not).

So if I found the game to be a technical masterpiece, but horribly offensive, I'd rate it something like a 1/10 - "No amount of excellent mechanics can make me able to enjoy something I find this offensive."

I would scrap the review myself.

If the offensive thing is so bad that the game goes from 10/10 material to 1, then it have to be something like neo-nazis making a game about gassing jews (and somehow receiving a massive budget for it).
In which case I would argue not give the game any time in the spotlight at all. Negative attention is also attention.

Alternatively if the game were about hard gay pornography, or something else than I'm really not into, then I'd still give it a 10/10, if I had to review it and put in bolded text that the game is only for guys who are into that stuff.
It would be really petty and unfair to slam the game in such scenario.

In both scenarios I'm not reviewing the game for myself, but have my audience in mind.