The simple solution to the Metacritic problem

[Vanguard_Ex]

This is possibly the stupidest idea I've heard since "Battleship movie". You can't compare games that are completely different and make one definitively better than the other, and things get more confusing when you've reviewed hundreds of games. What order would you put, say, Gears of War 2, Oblivion, Rock Band 2, Civilization 4, and Forza Motorsport 3. Not by your favorite, but by best, so your favorite genres and biases can't be a factor. Now imagine doing that with hundreds of games, all lined up in some definitive order of "best" to "worst".

Plus, this doesn't "solve" anything in the first place. The "problem with metacritic" is that people treat metacritic like it's fact so if one game gets a higher score than another, that makes it automatically better. Forcing reviewers to actually have to say which is "better" would make things worse.

The point is it's apples and oranges; different games are good in different ways, so you can't compare them in one long order. Also, it just wouldn't make any sense to say "This game gets a grade of 57/256 games." Can I stop ranting now? Do we understand why this idea is bad and you should feel bad?

EDIT: Whoops, I thought OP was talking about professional review scores, not user scores. Disregard all of that...

Wow man, seriously?

Fuck this place, really.

 

[veloper]

waaaaaaaaaaaay to complicated.
the best and only way to have the unwashed masses people assign ratings is the youtube option of thumbs up or down because nobody cares about one review out of hundreds, all people look at is the average score. and if having a number at the end of your review is so important to you feel free to ad one after the last sentence.

Wouldn't stop things like zero-bombing. Hundreds of people would just rate up the zero reviews.

except nothing will ever stop zero-bombing, if you give people the option to rate stuff and they want to use it as a hate machine they will find a way

Actually, making things more complicated for the voter, may just remove some of the poorly thought out user ratings that people here hate in metacritic.
Registering new alts to bomb a game only let's you put them at #1 with zero relevancy.

Displaying the current ranked list in front of the voter, may even help putting games in perspective: even if the voter is a hater, there's always something more hated.

I think the idea has alot of merit.
Rating if done consistently, is similar to ranking, only with fewer positions and possible gaps. A better game should never get a worse score and a reviewer who cannot even compare similar games has no business doing reviews anyway.
No game exists in a vacuum and even reviewing a game on it's own merits, only takes the originality factor of who did it first, out of the equation. Excellent and poor are still relative only to experiences with other games.

Ranking has alot of beneifts and no real downsides:
1. the top and bottom ranks are only extreme, for users who have already voted alot of games, so trolls registering new alts to bomb a particular game have no impact

2. it takes effort to build up a personal list form scratch

3. ranking is a transparant system and people who make random lists with a target at the end, can easily be recognized and removed

4. a good reviewer should always suggest better alternatives for a poor game and this scheme makes that an automatic

5. people who have played more games should have a better perspective on games and this scheme makes their votes more weighted.


The only thing that needs to be worked out is the categories. Comparing shooters to shooters works, but comparing them to puzzle games, not so much.

 

[maninahat]

Frankly, it is a terrible idea.

1. How easy is it to compare two seperate games in two totally different genres?
2. And what happens when a reviewer reviews hundreds of games? I can't imagine it being very simple or practical.

1. How is it not easy? Either you like A better than B, or B better than A. It's no more difficult than the current system.
2. You mean because the list would be too long to fit on one screen?

1. It ain't easy for the reason I stated: if the games tell very different stories, have totally different gameplay, from different periods in time, and are in totally different genres, then how exactly is it helpful for reviewers to throw them into some kind of order? Is Galaxian better than COD4? Is Portal better than Pac Man? Instead of just judging a game on its own merits, this system requires gamers to judge a game on another, totally different game's merits as well.
2. Are we just including 10 games, or 1000 games? If a game weighs in at number 789, could it be considered a shitty game, or is it just a very enjoyable game that happens to be competing against 788 marginally better games? You can't tell from the ranking. Basically this system encourages people to only buy stuff in the top few, and disregard anything that is too low in the list. That's going to suck for niche genres (like flight simulators) because they will inevitably get ranked lower by all the reviewers who prefer their FPSs over Sims.

The whole point of a review is to tell you if the game is good, not if it is better. If a reviewer wants to come up with a top 10 favourite games or recommend similar titles to the one being reviewed then fine, but they should only be doing it to suppliment their ordinary reviews.

 

[tehroc]

heres a thought... you could ignore user reviews. personally i dont see why they are there, they are obviously not accurate.

User scores are there cause it's entirely too easy to buy good critic ratings. 99% of every professional game reviewer will be corrupted, it's all about free games and playing them before release for those types.

 

[Athinira]

The simple solution to the Metacritic problem: READ the reviews instead of reading the score.

A high score doesn't mean you'll like a game.
A low score doesn't mean you'll hate a game.
A terribly written user review full of hate and with a 0 out of 10 is hardly going to justify the games true quality.
A well written critic review full of praise and with tendencies to discard faults is likely rating the game higher than it deserves.
A well-written, almost completely non-biased review with many good points is not going to help you if it's a genre you don't like or you disagree with the points being made.

TL;DR: As always, the simplest solution is actually using your brain.

 

[veloper]

Uh . . . no.
So if I play a completely shit game, and only that game, then that's my #1 ranked game?

In that case you shouldn't be handing out scores at all.
If all you play is one shitty game, you're not the person to recommend games to anyone.

 

[teebeeohh]

Actually, making things more complicated for the voter, may just remove some of the poorly thought out user ratings that people here hate in metacritic.
Registering new alts to bomb a game only let's you put them at #1 with zero relevancy.

Displaying the current ranked list in front of the voter, may even help putting games in perspective: even if the voter is a hater, there's always something more hated.

I think the idea has alot of merit.
Rating if done consistently, is similar to ranking, only with fewer positions and possible gaps. A better game should never get a worse score and a reviewer who cannot even compare similar games has no business doing reviews anyway.
No game exists in a vacuum and even reviewing a game on it's own merits, only takes the originality factor of who did it first, out of the equation. Excellent and poor are still relative only to experiences with other games.

Ranking has alot of beneifts and no real downsides:
1. the top and bottom ranks are only extreme, for users who have already voted alot of games, so trolls registering new alts to bomb a particular game have no impact

2. it takes effort to build up a personal list form scratch

3. ranking is a transparant system and people who make random lists with a target at the end, can easily be recognized and removed

4. a good reviewer should always suggest better alternatives for a poor game and this scheme makes that an automatic

5. people who have played more games should have a better perspective on games and this scheme makes their votes more weighted.


The only thing that needs to be worked out is the categories. Comparing shooters to shooters works, but comparing them to puzzle games, not so much.

so i need to play 10+ shooters for my tf2 rating to actually mean something? Even if it's the only shooter i play? that would make my vote for the game almost irrelevant even if i write a good, well done review full of crunchy points that have a lot of merit.
and wouldn't this lead to massive flaming, not for my rating but because of the way i categorize the games i review?
also: if i just punch in the name of a game what will i see and how will this help me decide if a game is good? the system seem to needlessly complicated to just give me a number and even if it did i doubt it would be possible for the average user to determine where this number comes from.

and wouldn't this discourage people from participating because you are basically putting up a huge sign saying: "you must devote this much time to participate", creating an oligarchy of rating that will eventually make metacritic irrelevant except for people who are too lazy to calculate an average out of all the professional review sites themselves?

 

[veloper]

Actually, making things more complicated for the voter, may just remove some of the poorly thought out user ratings that people here hate in metacritic.
Registering new alts to bomb a game only let's you put them at #1 with zero relevancy.

Displaying the current ranked list in front of the voter, may even help putting games in perspective: even if the voter is a hater, there's always something more hated.

I think the idea has alot of merit.
Rating if done consistently, is similar to ranking, only with fewer positions and possible gaps. A better game should never get a worse score and a reviewer who cannot even compare similar games has no business doing reviews anyway.
No game exists in a vacuum and even reviewing a game on it's own merits, only takes the originality factor of who did it first, out of the equation. Excellent and poor are still relative only to experiences with other games.

Ranking has alot of beneifts and no real downsides:
1. the top and bottom ranks are only extreme, for users who have already voted alot of games, so trolls registering new alts to bomb a particular game have no impact

2. it takes effort to build up a personal list form scratch

3. ranking is a transparant system and people who make random lists with a target at the end, can easily be recognized and removed

4. a good reviewer should always suggest better alternatives for a poor game and this scheme makes that an automatic

5. people who have played more games should have a better perspective on games and this scheme makes their votes more weighted.


The only thing that needs to be worked out is the categories. Comparing shooters to shooters works, but comparing them to puzzle games, not so much.

so i need to play 10+ shooters for my tf2 rating to actually mean something? Even if it's the only shooter i play? that would make my vote for the game almost irrelevant even if i write a good, well done review full of crunchy points that have a lot of merit.

Yes it would. Maybe the cut-off point wouldn't be at 10, but it makes sense that if someone is to recommend a shooter to the public, he's played more than just one.

and wouldn't this lead to massive flaming, not for my rating but because of the way i categorize the games i review?

I don't think the users would get to categorize the games themselves.
This could lead to the situation where someone might want to expand his shooter list with Mass Effect 2, but cannot because ME2 is listed as role-playing on metacritic and the site would get some flack for it. So be it.

also: if i just punch in the name of a game what will i see and how will this help me decide if a game is good? the system seem to needlessly complicated to just give me a number and even if it did i doubt it would be possible for the average user to determine where this number comes from.

This actually very simple: a system can normalize all the user rankings for you, above the cutoff point, onto a single percentile scale.

and wouldn't this discourage people from participating because you are basically putting up a huge sign saying: "you must devote this much time to participate", creating an oligarchy of rating that will eventually make metacritic irrelevant except for people who are too lazy to calculate an average out of all the professional review sites themselves?

And why would you have it any other way? What value is the recommendation of some user who doesn't even make an effort?

 

[teebeeohh]

so i need to play 10+ shooters for my tf2 rating to actually mean something? Even if it's the only shooter i play? that would make my vote for the game almost irrelevant even if i write a good, well done review full of crunchy points that have a lot of merit.

Yes it would. Maybe the cut-off point wouldn't be at 10, but it makes sense that if someone is to recommend a shooter to the public, he's played more than just one.

so if i write a bunch of good reviews for let's say RPGs and i buy one shooter and praise it for good gameplay, amazing story, interesting characters and motivation MP, my word has less merit than the word of the guy who wrote "OMG COD IS TEH ROZZ0Rs" as a review for every CoD game ever just because he wrote more reviews?
shouldn't i be able to recommend a game because it have played tons of games and can tell if a game is good an engaging without having played others like it?

also: if i just punch in the name of a game what will i see and how will this help me decide if a game is good? the system seem to needlessly complicated to just give me a number and even if it did i doubt it would be possible for the average user to determine where this number comes from.

This actually very simple: a system can normalize all the user rankings for you, above the cutoff point, onto a single percentile scale.

yeah my problem with that is that right now it's clear where the score comes from (total score of all reviews divided by number or reviews) and while i get how you would get a normalized score out of the ranking system it wouldn't be as easy to get as the current system.

and wouldn't this discourage people from participating because you are basically putting up a huge sign saying: "you must devote this much time to participate", creating an oligarchy of rating that will eventually make metacritic irrelevant except for people who are too lazy to calculate an average out of all the professional review sites themselves?

And why would you have it any other way? What value is the recommendation of some user who doesn't even make an effort?

because sometimes "sucked, controls" is all i need to know. Besides, ranking seems to encourage writing a lot of reviews, not writing good ones. In order to give my reviews meaning i could just write 10 one-line reviews and be done with it and that's what people will do if one review is important to them. It also sounds incredibly elitist to say that unless people have spend X hours writing reviews they will be ignored, and i am a PC gamer.

Why not split the user reviews into a review and rating section? in the rating section the trolls and hatebombers can thumb up/down a game to get a nice big green/red bar and in the actual review section people can write proper reviews using the ranking system (which i think is a great idea, i just don't think it will work well metacritic).

 

[Jandau]

NOTE: THIS IS A SUGGESTION FOR USER SCORES ONLY, not critic scores

Hard to imagine this hasn't been said before, somewhere, but anyway...

Instead of allowing users to assign a score, only allow them to assign a rank (relative to games they've previously ranked) and derive the score from that. So if I have ranked 9 games on metacritic then the highest would be scored let's say 9, the next 8, and so on down to the lowest, which would get 1. When I decide to rank a 10th game, I can't assign a score to it, I can only say where it ranked relative to the other 9. Metacritic could then adjust the scores for all 10 of my games accordingly. It's not a perfect solution, but in many ways it'd be an improvement on what they have now. Discuss.

Edited to add:
Apparently this requires further explanation.
You don't have to choose 10 games. You can have 1, or a million. You just say what order they're in. So if you have 10 games ranked, and you buy an 11th and want to rank it, then you have to say whether the new game is 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th, 9th, 10th, or 11th on your list of games.

So, if I were to review/rate 10 games I really like, I would end up lowering the score for most of them? Yeah, that makes PERFECT sense...

 

[RanD00M]

Well the best solution is to disregard Metacritic completely and read reviews at other places.

 

[valleyshrew]

This is literally the single worst idea I've ever seen from the gaming community. We generally only play good games, unlike reviewers. So you'd end up having to give 1/10 to a game that got 8/10 simply because you're not going to play 7/10 and below games as there isn't enough time to waste on them. In my last 10 games, that'd probably be portal 2 getting 1/10. Really great idea!

 

[Treblaine]

NOTE: THIS IS A SUGGESTION FOR USER SCORES ONLY, not critic scores

Hard to imagine this hasn't been said before, somewhere, but anyway...

Instead of allowing users to assign a score, only allow them to assign a rank (relative to games they've previously ranked) and derive the score from that. So if I have ranked 9 games on metacritic then the highest would be scored let's say 9, the next 8, and so on down to the lowest, which would get 1. When I decide to rank a 10th game, I can't assign a score to it, I can only say where it ranked relative to the other 9. Metacritic could then adjust the scores for all 10 of my games accordingly. It's not a perfect solution, but in many ways it'd be an improvement on what they have now. Discuss.

Edited to add:
Apparently this requires further explanation.
You don't have to choose 10 games. You can have 1, or a million. You just say what order they're in. So if you have 10 games ranked, and you buy an 11th and want to rank it, then you have to say whether the new game is 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th, 9th, 10th, or 11th on your list of games.

Nice idea, but it doesn't belong on metacritic.

My suggestion: file for a patent for the system and try it on your own website. There is no reason for this to tie in with metacritic which IS an "X/10" website, it just is.

 

[Jitters Caffeine]

I would say this is FAR from a simple solution to ANY problem. But I think I know what you're going for. A problem I see though is what happens if you have a LONG list of games you score? Do your scores count LESS if your list is longer? Like if you have a list of 10 games, and you put a game in the 10th place, that means it's bad. But if you have a list of 100 games and you put a game at 10, that would make it fairly good. Wouldn't that complicate your system a little?

 

[veloper]

so i need to play 10+ shooters for my tf2 rating to actually mean something? Even if it's the only shooter i play? that would make my vote for the game almost irrelevant even if i write a good, well done review full of crunchy points that have a lot of merit.

Yes it would. Maybe the cut-off point wouldn't be at 10, but it makes sense that if someone is to recommend a shooter to the public, he's played more than just one.

so if i write a bunch of good reviews for let's say RPGs and i buy one shooter and praise it for good gameplay, amazing story, interesting characters and motivation MP, my word has less merit than the word of the guy who wrote "OMG COD IS TEH ROZZ0Rs" as a review for every CoD game ever just because he wrote more reviews?
shouldn't i be able to recommend a game because it have played tons of games and can tell if a game is good an engaging without having played others like it?

For the purpose of aggregate scores, yes, it has less merit.
The reason for this is that while your review may be an excellent piece of prose, there's no way for a system to quantify this value. This is no different under the current system. Under the current system, you could have just copy/pasted something(anything) and still be a single vote bomber or do a 10/10 for a company. Nolonger.

What does change here is that your COD fan will have to rank the COD games in order of preference, instead of giving all of them the maximum score. It's not unreasonable to suppose that the aggregate of all COD fans may actually be a fairly good judge on the relative quality between those sequels.

also: if i just punch in the name of a game what will i see and how will this help me decide if a game is good? the system seem to needlessly complicated to just give me a number and even if it did i doubt it would be possible for the average user to determine where this number comes from.

This actually very simple: a system can normalize all the user rankings for you, above the cutoff point, onto a single percentile scale.

yeah my problem with that is that right now it's clear where the score comes from (total score of all reviews divided by number or reviews) and while i get how you would get a normalized score out of the ranking system it wouldn't be as easy to get as the current system.

It could still be just as clear where the score comes from. There's various way to do weigh the scores and one simple way is to display the following formula: user_score = 100% * (sum(points)/sum(listlenght))
What happens here is points get assigned inversely to the rank, so if a user has 20 games reviewed, the #1 game gets 20 points. Now someone who has reviewed alot of games makes a bigger impact, which is what I wanted.

and wouldn't this discourage people from participating because you are basically putting up a huge sign saying: "you must devote this much time to participate", creating an oligarchy of rating that will eventually make metacritic irrelevant except for people who are too lazy to calculate an average out of all the professional review sites themselves?

And why would you have it any other way? What value is the recommendation of some user who doesn't even make an effort?

because sometimes "sucked, controls" is all i need to know. Besides, ranking seems to encourage writing a lot of reviews, not writing good ones. In order to give my reviews meaning i could just write 10 one-line reviews and be done with it and that's what people will do if one review is important to them. It also sounds incredibly elitist to say that unless people have spend X hours writing reviews they will be ignored, and i am a PC gamer.

Why not split the user reviews into a review and rating section? in the rating section the trolls and hatebombers can thumb up/down a game to get a nice big green/red bar and in the actual review section people can write proper reviews using the ranking system (which i think is a great idea, i just don't think it will work well metacritic).

I though you were arguing for long reviews instead of "sucked,controls", but it doesn't matter. The "sucked, controls" guy may be still talking out of his ass; you don't know him afteral.
All you can get out of the user scores is hoping the unreasonable extremes average eachother out and hope to get something useful out of that, which is how metacritic currently operates already. A single user review by itself means less than a bought review on a big online gaming mag.

I reckon the OP's scheme would do a better job than the what we got now, exactly because it limits the impact of single votes.
That and because it would force a gradual scale on the users, instead of promoting all 10s or zeros.

 

[Epic Fail 1977]

Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.

Thank you for trying. It's been a while since I posted a new thread at the Escapist. Now I remember why...

 

[Vanguard_Ex]

Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.

Thank you for trying. It's been a while since I posted a new thread at the Escapist. Now I remember why...

Because now more than ever our community is a venomous den of sarcastic putdowns and scoffs? Yeah, that's why I'm hesitant to say a fair few things.
And you're welcome mate.