The simple solution to the Metacritic problem

Recommended Videos

Epic Fail 1977

New member
Dec 14, 2010
686
0
0
Glass Joe the Champ said:
Now, if you asked me to "officially" say which one is better, I couldn't do it because it depends on whether you're looking for an RPG or a racing game.
This is what I was talking about in the post above. I always thought that your user score was for you to say which one you think is better. Sorry to break it to you, but you're not an "official" reviewer (and even if you are, your review should then be in the critics section, not the users section).
 

pppppppppppppppppp

New member
Jun 23, 2011
1,519
0
0
Guy Jackson said:
Glass Joe the Champ said:
Now, if you asked me to "officially" say which one is better, I couldn't do it because it depends on whether you're looking for an RPG or a racing game.
This is what I was talking about in the post above. I always thought that your user score was for you to say which one you think is better. Sorry to break it to you, but you're not an "official" reviewer (and even if you are, your review should then be in the critics section, not the users section).
Oh....user score.....well, don't I feel like a jackass now.

Nevermind, I thought you were talking about professional reviewer scores.

So yeah, this is a decent idea then. (retreats back to the corner in shame)
 

beniki

New member
May 28, 2009
745
0
0
Glass Joe the Champ said:


This is possibly the stupidest idea I've heard since "Battleship movie". You can't compare games that are completely different and make one definitively better than the other, and things get more confusing when you've reviewed hundreds of games. What order would you put, say, Gears of War 2, Oblivion, Rock Band 2, Civilization 4, and Forza Motorsport 3. Not by your favorite, but by best, so your favorite genres and biases can't be a factor. Now imagine doing that with hundreds of games, all lined up in some definitive order of "best" to "worst".

Plus, this doesn't "solve" anything in the first place. The "problem with metacritic" is that people treat metacritic like it's fact so if one game gets a higher score than another, that makes it automatically better. Forcing reviewers to actually have to say which is "better" would make things worse.

The point is it's apples and oranges; different games are good in different ways, so you can't compare them in one long order. Also, it just wouldn't make any sense to say "This game gets a grade of 57/256 games." Can I stop ranting now? Do we understand why this idea is bad and you should feel bad?
Wait... so a system that shows the opinion of the user is bad?

It does make sense. The raw list could then be filtered by genre, or year or whatever you want to get rid of the apples and oranges problem. And even then, the original list would give a good indicator as to what the most popular genre is.

And you know... forcing critics to think critically about the order of their games is kind of their job. That's why we all like to watch Top Ten lists!

Not sure why it would be weird to see the score in relation to other games. If it was near the top of list, I'd assume good, if near the bottom, I'd assume bad. It's actually a little more honest than an arbitrary number.
 

tippy2k2

Beloved Tyrant
Legacy
Mar 15, 2008
14,870
2,349
118
beniki said:
tippy2k2 said:
Guy Jackson said:
tippy2k2 said:
In the second part of my post, I stated that even if you forced me to choose ten games (which your original post STILL says you don't have to do that, though you are saying here that you do have to do that)
No, you don't HAVE to rate 10 games, but if you do then your highest score will be higher and the lowest will be lower (the bold part being of interest to bombers).

tippy2k2 said:
I hit TAB on the top ten items and rate them whatever. I suppose if you really need me to test this I can but I'm thinking a minute at most.
So let's say we've added a minute and thirty seconds (plus the time it takes to make a new email account) to the time it takes to review-bomb a game with a single "zero" score. I'd say that'll reduce the number of bombs. Sure, some will still do it, but less.

tippy2k2 said:
And yes, learning a new system is a hassle. You ever watch the hissy fits people throw when Facebook adjusts anything? This is adding another step in the process that does not hurt bombers, so yes, adding another step is a hassle to your legit users.
What extra step? For the legit users, I mean? I don't see an extra step.
Your entire new process is one big extra step.

Current way of doing it:
I find my game of choice
I click 1-10

Your way of doing it:
I find my game of choice
I click 1-10
If I now have a second game, I have to slide that game wherever, repeat until I have ten

Notice that it is one extra step compared to what I have to do now.

According to your system, if I legitimately think that MW3 is the worst game ever and deserves a 0, I have to choose ten more games to put on my list in order to reflect this. If I'm a bomber, I don't care and will just fuck up ten random scores to smack MW3 down. If I'm a legit user, I now have nine more games that I'm going to have to review in order for my true opinion to be heard. Now a legit user has nine extra steps to take.

The mistake you're making here is assuming that adding two minutes of a troll's review is going to stop them from bombing. It won't. All you are doing is smacking legit users.
... What?

1) Open Account
2) Write a review for a game.
3) Write a review for another game.
4) Say which one is better.

I'm really not sure what your problem is... the idea is to simply remove the idea of a score and just put the games in order from bad to good. No extra steps, and only gets complicated if you have maybe 50+ games, and you can't decide where the new one should go.

If you are a bomber, you'd have to input a lot of games to make a difference. Make a new account with just one game, and that game will be considered to be the best by the system.

You have Metacritics biggest problem; too focused on scores.

I think it's a good idea. Sure it may be dificult to make comparisons between RTSs and FPSs, but surely that would just show what the user likes. That's good data to have... you could finally show which game genre is considered the best, without the use of market data.
I don't know what my problem is either because I guess I don't understand his system. All I see is a system that makes a legit user do more work to put their reviews in but not enough work to make a bomber turn back.

It might just be my misunderstanding of his idea so I'll just sit here in the corner and watch.
 

Epic Fail 1977

New member
Dec 14, 2010
686
0
0
Glass Joe the Champ said:
Guy Jackson said:
Glass Joe the Champ said:
Now, if you asked me to "officially" say which one is better, I couldn't do it because it depends on whether you're looking for an RPG or a racing game.
This is what I was talking about in the post above. I always thought that your user score was for you to say which one you think is better. Sorry to break it to you, but you're not an "official" reviewer (and even if you are, your review should then be in the critics section, not the users section).
Oh....user score.....well, don't I feel like a jackass now.

Nevermind, I thought you were talking about professional reviewer scores.

So yeah, this is a decent idea then. (retreats back to the corner in shame)
Crap, another edit to the OP. I wonder how many other people I confused :S
 

Dutchy115

New member
Nov 7, 2011
81
0
0
There isn't any easy total solution to the problem, but there are a few minor adjustments that could be made to make the system fairer.
For example, for games that are widely known to have nemesis titles both with unhealthy, endlessly-dedicated extremist fan-bases (such as MW3 + BF3), users will not be permitted to give one game a 10 and the other a 0, as the only real explanation for such action is an extremely biased p.o.v on the behalf of the reviewer, the fact is, if you check the accounts of user?s reviewing on both game?s pages, they tend to have all given their preferred game a 10 and its competitor a 0. Alternatively, users could be given the option of flagging reviews for deliberate score inflation or deflation (i.e. A ?0? review for MW3 with the text ?AGDSGFDHSFGDSHDAGSHSAGFDSFGDSAHFD? or a ?O? review for MW2 with the text ?I?m appalled this game has such a high user-score, hopefully this will bring it down a bit?) this would also work for games given a ?10? without sufficient justification.
My Metacritic username is DarkEnergon, I like to think I review all games/movies fairly, with two exceptions, my reviews for Transformers: Dark of the Moon and MW3, which I gave a 9 and a 10 respectively, which upon current analysis, were somewhat biased or at the very least blinded by all the clap-trap reviews hitting the pages, I intend to change these scores a.s.a.p.
 

Jaime_Wolf

New member
Jul 17, 2009
1,194
0
0
Though it doesn't solve the biggest problems, this is still a really nice idea that lends itself well to exactly the sort of thing most people use reviews for - trying to decide whether a game is worth buying or trying to decide which of a small list of games is most worth buying.

Reviews are already treated as ordinal data, despite not being gathered in that way, and this would ensure that the data actually IS ordinal. It makes a lot of potentially useful data analyses much easier too.

This would not, however, solve the larger problems with Metacritic. People could still put games on the bottom of the list simply out of spite/fanhate/etc and, if scores were actually weighted based on the number of titles in a list (being on the bottom of a thousand is worse than being on the bottom of ten), people could use that to unfairly influence the system.

What Metacritic needs to do is normalize scores across reviewers/accounts. So if you consistently try to manipulate the rating by scoring things exceptionally low or high, it won't matter at all.
 

Talux

New member
Apr 9, 2008
39
0
0
Why not stop worrying about scores or ranks and actually, you know, read or watch a review?

CoD 7 or BF3 getting a nine or a ten is completely meaningless to me. It's just an arbitrary number. The same goes for all the ridiculous review bombing. A lot of the time I just wish people would cut out review scores entirely or at least bring them down to fairly simple star ratings. Saying that CoD7 is a '9' is completely meaningless to me compared to someone describing how tight the controls are, how well balanced the weapons and maps are, whether or not it's innovative at all, whether the single player campaign is worth the effort and so on.
 

lastoftheline

New member
Nov 20, 2011
8
0
0
I think the issue is less with how metacritic works (i think it's beautiful) and morw with a strange view that gamers have, especially thoe who queued up at midnight on the night of release of MW3.

It seems they expect that the game almost belongs to them, and have thus gotten up-set that Infinity ward, and moreover activision have deemed to release pretty much the exact same game that they have the previous two times. Is it really unexpected considering the fact THEY ALL STILL BOUGHT IT!?!?!

Why fix what makes you a billion dollars?

I'm sure MW3 stands up as a game on it's own quite well, why everyone is having a fit about it don't know. Some gamers need to get over themselves, beause I could bet a large number of those who have given it a 1 are still playing it and haven't seen the sun for three weeks!

/rant
 

Yopaz

Sarcastic overlord
Jun 3, 2009
6,092
0
0
Guy Jackson said:
MiracleOfSound said:
That doesn't make sense because the scores would only be relative to other stuff.

You could have a gem of a game that would be ranked badly relative to other good games, or an average game getting a great rank because it's in a group of games that are complete ass.
If the gem really is so great then why did you rank it so low?
And if all your games are complete ass in your opinion (which is what your metacritic score is supposed to reflect) then why are you even a gamer?
The ranking system doesn't indicate how good a game is. If I have games ranked in the following order. (Not an actual list, just a random compilation of good games to make a point)
Legend of Zelda Skyward Sword
Skyrim
Batman Arkham Asylum
Saints Row The Third
Deus Ex Human Revolution
Alone In The Dark
Duke Nukem Forever
With this list it would indicate that Alone In The Dark is almost as good as Deus Ex. I have played both games and trust me. They are not equally good. Where Deus Ex is a gem Alone In The Dark's only selling point is that you can skip through the story if you don't want to play it. This system would not work.
 

teebeeohh

New member
Jun 17, 2009
2,896
0
0
waaaaaaaaaaaay to complicated.
the best and only way to have the unwashed masses people assign ratings is the youtube option of thumbs up or down because nobody cares about one review out of hundreds, all people look at is the average score. and if having a number at the end of your review is so important to you feel free to ad one after the last sentence.
 

Rednog

New member
Nov 3, 2008
3,567
0
0
The simplest solution to metacritic is to simply just get rid of it.
The system is completely flawed and with various legit reviewers not actually working on the same rating scale leads to problems and then users are well....users, and you really can't solve people's stupidity.
 

The Woolly One

New member
Nov 25, 2010
47
0
0
The way to do it would be to delete user reviews that are too much higher or lower than the critic reviews. Scores that fall outside of this range would be ignored. For example, if critics rate a game as 6/10 then the user scores should be expected to fall between about 4 and 8, with a few outliers obviously, but statistically the majority ought to be around there so scores outside of that kind of range seem suspect. This would also stop people 'zero-bombing' good games out of silly reasons.

For example, people who gave Portal 2 zeroes were wrong. It wasn't their opinion, they were just wrong - zero represents a game that is entirely unplayable and cannot really be described as a game. They gave it a zero for things like day one DLC despite the fact that the game itself was amazing. Likewise with MW3 - people who gave it zeroes because it was the same as MW2 were again wrong. It is not unplayable, it is not broken. I don't think it should be rated as high as it is - I've played about 24 hours worth and its not significantly different to MW2 either graphically or gameplay wise. Spec ops are good, and survival is great fun, but it should lose marks for innovation. But it is a far better game than the users say - the gameplay is rivalled only by BF3, so how they can rate it as zero is beyond me.

Also, anyone who mentions the word 'average' in their review but gives a score other 4, 5 or 6 should have their review deleted or their score changed - you can't say a game is average and then give it an appalling or an amazing score.
 

The Woolly One

New member
Nov 25, 2010
47
0
0
teebeeohh said:
waaaaaaaaaaaay to complicated.
the best and only way to have the unwashed masses people assign ratings is the youtube option of thumbs up or down because nobody cares about one review out of hundreds, all people look at is the average score. and if having a number at the end of your review is so important to you feel free to ad one after the last sentence.
Wouldn't stop things like zero-bombing. Hundreds of people would just rate up the zero reviews.
 

piesandwich

New member
Jul 9, 2009
10
0
0
I really like this idea. It would be really interesting to see how the results differ from normal ratings.
Everyone seems to be getting upset over the idea of comparing games from different genres, which I can understand. But if everyone was ranking games based on which ones they LIKED the best rather than some pressure to conclude the definitive ?best game ever? then you can rank them based on your opinion without any worry.
I think the score that would be produced from a system like this would be pretty fair, so long as people rank a good share of crappy games so the mediocre ones landed in the middle bracket rather than the bottom.
 

teebeeohh

New member
Jun 17, 2009
2,896
0
0
The Woolly One said:
teebeeohh said:
waaaaaaaaaaaay to complicated.
the best and only way to have the unwashed masses people assign ratings is the youtube option of thumbs up or down because nobody cares about one review out of hundreds, all people look at is the average score. and if having a number at the end of your review is so important to you feel free to ad one after the last sentence.
Wouldn't stop things like zero-bombing. Hundreds of people would just rate up the zero reviews.
except nothing will ever stop zero-bombing, if you give people the option to rate stuff and they want to use it as a hate machine they will find a way
 

Woodsey

New member
Aug 9, 2009
14,553
0
0
Alternatively, people could just read the fucking words, and understand that any sort of ranking/score/one-word summation at the end of a review is only there to give the idea of a ball-park consensus of people's opinions on a game.

The simple solution to the Metacritic problem would be to blow it up.
 

Vanguard_Ex

New member
Mar 19, 2008
4,687
0
0
Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.