Which Outliers are Mistakes / Clickbait and which are common for the Publication?

[ran88dom99]

I have been working on a project to help users find publications with similar opinions. Outliers (technically defined as observations made by systems different than the rest of the data) can throw the algorithms off and the greater they are the greater the throwing. So an outlier for my purposes is a review score that is uncharacteristic of the publication. Just in case the score must be outlying for the game as well. Ironically, if a publication regularly makes factually inaccurate reviews, or sends people who hate the genera to review games from it then that revscore is not an outlier for them. It would be pointless of me to try to ask about people's knowledge of publications. Instead I ask if you know any of these to be naked clickbait, or as the case for Natural Selection 2 was, written by a random freelancer who never contributes to the publication again. Factualy Inaccurate Review [https://kotaku.com/5960657/metacritic-refuses-to-pull-negative-review-that-gamespot-admits-was-factually-inaccurate]

I would describe the common review process as decent but herding-instinct-y. Outliers can arise when the reviewer is simply not a sheep. While some outliers immediately seem excessive, one of the few games I can prove was grossly miss-rated ( Darwinia [http://strategywiki.org/wiki/Darwinia/Walkthrough]) has outliers very near its real worth.
Why Darwinia is so misrated: MS 84 Median 88. Based on letsplays, the game has an interesting but simple aesthetic and no gameplay. Never mind the pathfinding, though this game is pretty old and it's lets-plays are probably of the latest version. You move your guys to a spot then click on the red stuff and the guys shoot it. That is it. That is all. There is no challenge. You can not lose. There is nothing to learn or become skilled at. There is no variety. None of the publications rating it over 70 mention the complete lack of gameplay and most call it an RTS of all things. For comparison AE's review of Diablo3 [http://www.ae-infinite.com/2012/06/review-diablo-iii-pc.html] is reasonable but the score drop is overenthusiastic. Giving The Witcher 2 a score of 55 is sheer lunacy.

Please look at the outliers bellow and, if you know the game, post weather the outlier is crazy clickbait, justifiable, or a better appraisal of the games worth than the Media.

Here is the rest of the data. [https://drive.google.com/open?id=0B0Au73qlmej7UXhxcks2U2cxSkk] I will reduce the outlyingness of a score to it's closest non-outlier. The average score shift expected for Greatest Outliers is around 3.7 times more than Most Outliers. This data is adjusted with max at 100 and min at 40 because median of all games scored is at 70 and quartiles are at 80 and 60. A difference of 30 points is half the total range.

Common Outliers
Median, difference from median, game name, publication, cutoff for publication when its greater than 19 (different methods make different cutoffs).

90 -36.6 xbox/rallisport-challenge-2 [http://www.metacritic.com/game/xbox/rallisport-challenge-2/critic-reviews?dist=negative] operation-sports
92 -37 xbox-360/assassins-creed-ii [http://www.metacritic.com/game/xbox-360/assassins-creed-ii/critic-reviews?dist=negative] destructoid
90 -35 playstation-3/heavy-rain [http://www.metacritic.com/game/playstation-3/heavy-rain/critic-reviews?dist=negative] bigpond-gamearena
90 -36.6 playstation-3/heavy-rain metro-gamecentral -27.5
90 -41.6 xbox-360/need-for-speed-hot-pursuit gameshark
90 -35 xbox-360/the-witcher-2-assassins-of-kings [http://www.metacritic.com/game/xbox-360/the-witcher-2-assassins-of-kings/critic-reviews?dist=negative] gamer-limit
90 -30 xbox-360/the-witcher-2-assassins-of-kings darkstation
90 -31.6 xbox-360/halo-4 [http://www.metacritic.com/game/xbox-360/halo-4/critic-reviews?dist=negative] armchair-empire -25
90 -43.3 xbox-360/halo-4 quarter-to-three -43.3
90 -30 xbox-360&PS3/dead-space-2 [http://www.metacritic.com/game/playstation-3/dead-space-2/critic-reviews?dist=neutral] game-over-online
90 -30 playstation-3/demons-souls [http://www.metacritic.com/game/playstation-3/demons-souls/critic-reviews?dist=neutral] gamervision -20.5
90 -35 pc/diablo-iii armchair-empire

53.3 38.6 playstation-3/tony-hawk-ride [http://www.metacritic.com/game/playstation-3/tony-hawk-ride/critic-reviews?dist=positive] gamezone 22
55.3 34.6 xbox-360/tony-hawk-ride planet-xbox-360 23.1
55.3 36.6 xbox-360/tony-hawk-ride gamingtrend 22
85 -34 xbox-360/rage [http://www.metacritic.com/game/xbox-360/rage/critic-reviews?dist=negative] gameshark -27
85 -25 xbox-360/rage darkstation
85 -25 xbox-360/rage gamernode
85 -28.3 xbox-360/rage gamers-hell



Outlier detection methods from fewest to most outliers detected. The numbers next to each game are the difference from the median of the game.

**GRUBBS Highest difference**
-ds/new-super-mario-bros
playstation-3/duke-nukem-forever
-xbox-360/halo-4
-xbox/halo-2
xbox-360/army-of-two-the-devils-cartel
-xbox-360/dead-space-2
playstation-2/evil-dead-a-fistful-of-boomstick
playstation-3/start-the-party!
wii-u/game-wario
-xbox/rallisport-challenge-2
ds/spectrobes
-pc/diablo-iii
-playstation-2/final-fantasy-xii
-playstation-3/demons-souls
-xbox-360/the-witcher-2-assassins-of-kings
-pc/the-talos-principle
-playstation-3/fifa-soccer-13

**GRUBBS Most Outliers (2)**
-xbox-360/halo-4
ios/borderlands-legends vandal-online
playstation-4/submerged
-pc/escape-from-monkey-island
-ios/star-wars-knights-of-the-old-republic
-pc/fifa-15
xbox-360/facebreaker
-xbox/aggressive-inline
pc/great-invasions-the-darkages-350-1066-ad

**Medcup Highest Outliers**
34.3 playstation-4/the-amazing-spider-man-2
33.3 xbox-360/aliens-colonial-marines
-30.0 playstation-3/dead-space-2
-30.0 xbox-360/dead-space-2
-37.0 xbox-360/assassins-creed-ii
-36.7 playstation-3/heavy-rain
-36.7 xbox/rallisport-challenge-2
-35.0 pc/diablo-iii
-30.0 playstation-3/demons-souls
38.7 playstation-3/tony-hawk-ride
36.7 xbox-360/tony-hawk-ride
34.7 xbox-360/tony-hawk-ride

**Medcup Most Outliers (3)**
-35.2 psp/dissidia-final-fantasy
-27.3 xbox/mortal-kombat-deception
"+ - 20? xbox-360/dead-rising-2-case-zero
?+ - 20? pc/that-dragon-cancer
36.7 xbox-360/tony-hawk-ride

**Dixon Highest Outliers**
-36.7 psp/daxter
-39.0 xbox/doom-3-resurrection-of-evil
-36.0 playstation-2/grand-theft-auto-san-andreas
-37.0 xbox-360/assassins-creed-ii
-40.0 xbox/ninja-gaiden
-36.7 playstation-3/heavy-rain
-35.0 playstation-3/heavy-rain
-41.7 xbox-360/need-for-speed-hot-pursuit
-35.7 xbox/midnight-club-3-dub-edition
-35.0 pc/diablo-iii
40.0 wii/final-fantasy-crystal-chronicles-echoes-of-time
-35.0 xbox-360/the-witcher-2-assassins-of-kings

**Dixon Most Outliers (4)**
32.3 xbox-360/x-men-the-official-game
30.0 xbox-360/spider-man-3
-30.0 playstation-4/fallout-4

**IQR Highest Outliers**
40.0 playstation-4/entwined
-31.7 xbox-360/halo-4
-30.0 playstation-3/dead-space-2
-30.0 xbox-360/dead-space-2
-36.0 playstation-2/grand-theft-auto-san-andreas
-37.0 xbox-360/assassins-creed-ii
-41.7 xbox-360/need-for-speed-hot-pursuit
-36.7 xbox/rallisport-challenge-2
-35.0 pc/diablo-iii
-30.0 playstation-3/demons-souls
-30.0 xbox-360/the-witcher-2-assassins-of-kings
-35.0 xbox-360/the-witcher-2-assassins-of-kings
36.7 xbox-360/tony-hawk-ride
36.7 xbox-360/tony-hawk-ride

**IQR Most Outliers (4)**
-30.0 playstation-3/bayonetta
32.7 playstation-3/lair
+-27.5 wii-u/kirby-and-the-rainbow-curse
-25.0 wii-u/runbow
32.3 xbox-360/x-men-the-official-game
35.0 playstation-3/spider-man-3
30.0 xbox-360/spider-man-3
-34.0 xbox-360/rage
+-20.0 xbox-360/dead-rising-2-case-zero
-29.0 xbox-360/mafia-ii
-30.0 pc/dear-esther
-30.7 playstation-3/la-noire
-36.7 playstation-3/heavy-rain
40.0 playstation-2/driv3r
26.7 xbox-360/sonic-free-riders
36.7 xbox-360/tony-hawk-ride

**Chi-squared Highest Outliers**
40.0 playstation-4/entwined
-34.0 xbox-360/rage
-40.0 gamecube/killer7
-36.3 playstation-3/uncharted-3-drakes-deception
-37.0 xbox-360/assassins-creed-ii
-40.0 xbox/ninja-gaiden
-35.0 playstation-3/heavy-rain
-36.7 playstation-4/the-witness
-41.7 xbox-360/need-for-speed-hot-pursuit
40.0 xbox/driv3r
-36.7 xbox/rallisport-challenge-2
-35.0 pc/diablo-iii
-35.0 xbox-360/the-witcher-2-assassins-of-kings

**Chi-Squared Most Outliers (7)**
40.0 playstation-4/entwined
+-26.7 xbox-one/quantum-break
-34.0 xbox-360/rage
+-30.0 pc/dear-esther
-36.7 playstation-4/the-witness
-20.7 playstation-3/motorstorm-apocalypse
+-26.7 xbox-360/from-dust[/QUOTE]

 

[Avnger]

Outliers need to be accounted for in an algorithm, not tossed aside and forgotten. You've already removed any possible objectiveness from your project when your personal opinions on a game are deciding which scores you consider outliers. Basically, rethink your entire approach to the data set you have. Otherwise, you'll end up with with a guide that isn't valid at all because you manipulated the data to get the results you wanted.

 

[ran88dom99]

You've already removed any possible objectiveness from your project when your personal opinions on a game are deciding which scores you consider outliers.

And how could it be possible to decide objectively which of the 5 (or 10) methods above is the correct outlier identifier? I read the literature. There is no common professional consensus. http://strategywiki.org/wiki/Darwinia/Walkthrough

 

[CaitSeith]

Shouldn't this be in the Gaming Industry Discussion forums? I thought about the concept of a website where the users could cross-reference reviewers or Youtubers with games and their scores (or an positive/negative/neutral indicator for unscored ones) so gamers could find the ones who shared their preferences.

So, what's the purpose of finding Outliers?

 

[Phoenixmgs_v1legacy]

I'm sorry, but I find this project to be completely useless. How are you supposed to find a publication that shares your preferences when all publications (at least on Metacritic) all share the same opinion about pretty much every game? The vast majority of games (AAA games for sure) rarely have any negative reviews. Look at Final Fantasy 13 with only a single negative review when that was a major love/hate game. I'm guessing you'd consider that review to be an outlier considering it was a Jim Sterling review, plus he doesn't work there anymore. Game criticism is so bad that it's impossible to find a publication that can give you a decent chance of knowing if you'll like the game from their review. I don't think finding a publication is feasible even if game criticism was healthy because even with movie reviews, you find a reviewer here or there that you actually "get" something out of their reviews let alone an entire publication with several reviewers. The best you can find in gaming would be a YouTube reviewer or a very little known reviewer with a website, and you're not going to be able have easy access to a list of games they reviewed with corresponding scores as their reviews won't appear on your Metacritics or GameRankings.

There's lots of AAA games that have average scores in the 90s that I find to be average or just plain bad games including stuff like GTAV, Uncharted 4, Witcher 3, etc. That's not "lunacy", I don't find those games to be good. What publication has a negative review for any of those games let alone all of the them?

 

[ran88dom99]

Shouldn't this be in the Gaming Industry Discussion forums? I thought about the concept of a website where the users could cross-reference reviewers or Youtubers with games and their scores (or an positive/negative/neutral indicator for unscored ones) so gamers could find the ones who shared their preferences.

So, what's the purpose of finding Outliers?

I did not notice GID. For movies; Criticker TasteKid and Listal used to.

They mess up the algorithms. A half range (30 p) difference from the publication pretty much guarantees the publication will not be in the top 10.

I'm sorry, but I find this project to be completely useless. How are you supposed to find a publication that shares your preferences when all publications (at least on Metacritic) all share the same opinion about pretty much every game? The vast majority of games (AAA games for sure) rarely have any negative reviews. Look at Final Fantasy 13 with only a single negative review when that was a major love/hate game. I'm guessing you'd consider that review to be an outlier considering it was a Jim Sterling review, plus he doesn't work there anymore. Game criticism is so bad that it's impossible to find a publication that can give you a decent chance of knowing if you'll like the game from their review. I don't think finding a publication is feasible even if game criticism was healthy because even with movie reviews, you find a reviewer here or there that you actually "get" something out of their reviews let alone an entire publication with several reviewers. The best you can find in gaming would be a YouTube reviewer or a very little known reviewer with a website, and you're not going to be able have easy access to a list of games they reviewed with corresponding scores as their reviews won't appear on your Metacritics or GameRankings.

There's lots of AAA games that have average scores in the 90s that I find to be average or just plain bad games including stuff like GTAV, Uncharted 4, Witcher 3, etc. That's not "lunacy", I don't find those games to be good. What publication has a negative review for any of those games let alone all of the them?

It is better than nothing. If there does not exist a publication that is completely honest there will be publications more honest than the rest. Or with closer tastes to yours. I would hope, as this program gains popularity, those small scale publications would contribute their own data. Scraping is a pain.

By the way the program works. When I input lesser known 4X games and select publications based on how they focus on such things, the recommendation algorithm spits out brainy games and very strongly down rates shooters. Sure its just genera bias but as proof its very objective.