Realism carries certain advantages in making complicated games accessible. Take Skyrim, Red Dead Redemption, Assassin's Creed, Crysis, Half-Life, or Battlefield. All of these are very complex games with a lot of systems overlaid on each other in a very simulator-like way, with a lot of detail in how you interact with the world.
Half of why it works is the realism of the presentation and the relative grounding of the setting around you. When you pick up one of these games, you instantly slip into a mode where you start thinking about solving problems and approaching interactions the way you would in the real world--which means the designers don't have to spend a lot of time explaining things about how the world around you works.
Skinning animals in Red Dead or blacksmithing in Skyrim ends up taking very little to explain. In fact, you didn't even need it explained that there
were animals in the first place, did you? In Battlefield, the tactical roles of various weapons and vehicles is easy to recognize, because it's the same as it is in real life--though military hardware is admittedly a... niche interest. Finally, in Half-Life and Crysis, nobody needs physics explained or someone shouting over an intercom to help you recognize the role it might have in problem-solving. You see a buzzsaw laying on the floor, instantly recognize its use with Half-Life 2's gravity gun, and feel all the cleverer for it, giving a stronger sense of satisfaction.
In general I'd say that's why realism and more realistic settings for games have had a big surge of popularity for this console generation over previous ones--it makes very broad games a hell of a lot easier to swallow.
The diminishing returns, I'd say, come from games that're more narrow or abstract in nature.
Sonic the Hedgehog has no business looking realistic, for instance, the core concept behind it being that you're playing a pinball with legs who can also be a buzzsaw--two of the most fun round things imaginable in one character. So it's best to stylize in such a way as to make that even
more fun and reinforce the abstract concept. That's probably got a lot to do with why casino and arcade imagery works so well for him. The Kingdom Hearts and Final Fantasy games use very over-exaggerated action mechanics, and so an anime-like style makes them very easy to swallow. Devil May Cry, on the other hand, has very
detailed action mechanics, much like being the Tony Hawk of action games in the way it emphasizes combo-building, so it's more realistic in presentation to account for detail, but stylized enough to be recognizably anime-like to account for exaggerations and compliment the pro sports feel.
It all comes down to what you need the style to say about your game, really. Pick the qualities that stand out, then pick a style that compliments them. Sometimes, it happens to be realism.
DarthSka said:
like Zelda. Though it's a fantasy game, the realistic look in Twilight Princess
Twilight Princess? Realistic?
[img src="http://i108.piczo.com/view/2/w/b/g/0/1/m/1/i/j/a/9/img/i206256360_49560_4.jpg" /]
... Huge eyes, plasticy skin... No, still anime. I'd call it more "illustrative" or "painterly" than "realistic," though I'd still attribute those qualities to the Lord of the Rings-ish feel that you seem to get out of it (I never did - when in Lord of the Rings was there a little blue cartoon devil with a goofy hat riding around in someone's shadow?).
I guess it's more
detailed than the toon style, the proportions of characters are
often more accurate to life, but I never really got "realistic" out of this game. It does work, though, for a lot of the reasons realism worked for some of the games I outlined above, so I guess I gotta give you that. Personally I prefer Skyward Sword's more "Studio Ghibli" visuals, which simultaneously sell the believability and detail as well as the fantastical nature of its world.