You're making a lot of assumptions about what something /has/ to do. You complain that story in games prevents players from making choices, then say you couldn't you couldn't adapt Of Mice and Men without being forced to give people the ability to choose a 'good' ending. Which is it?
Games are by definition sets of rules, rules are by definition limitations of what you can do. Video games are a space in which you can do /some/ things but not /all/ things, no matter how wide it's rules or open-ended and emergent it's design there will always be limits. And not just limits of what the developers are capable of doing but what they want the player to be allowed to do. So how is story any different? Something will always be holding players back, if you really have to look at it that way, so how is making one of the things that's holding you back something entertaining and valuable in its own right a problem? And that's only if you choose to look at it as a restriction first and foremost anyways.
Just because one game does something one way doesn't mean that suddenly everything has to do it that way, there are no laws about what you can and can't do. You say that we can have either near infinite combinations in a narrative void (which is probably an exaggeration but that's semantics) or the narrow path of someone's narrative. Why does one of those options have to be inherently better then the other in every circumstance?
All you're saying is that there are games that exist without story, so therefor if it isn't always absolutely necessary it isn't ever needed in anything. Well, I guess we made silent films in black and white for years, so clearly you don't /need/ color or sound. It doesn't matter that these are tools skilled filmmakers can use to be powerful and evocative and create the kind of film that /they/ want to, the industry got by for 20 odd years with neither of them at all. Civilization has been enormously successful without any kind of gunplay, so I guess obviously there aren't any games that need shooting. It doesn't matter that it's intended to be a totally different kind of experience a lot of people enjoy and directly comparing the two yields very little, that franchise over there made it work so now that's the rule.
If you don't enjoy story in games in general, or at the very least have yet to see an example that really works for you, then that's fine. You're perfectly welcome to go enjoy the things that do work for you. But there are a lot of story driven games that I love, and the idea that they inherently don't work because of false comparisons of failing to live up to what other unrelated things are doing is ridiculous. I might as well say that I hate Halo because it's a terrible RPG. Well, yeah, it is a terrible RPG, that's why it doesn't say RPG on the box.
As one last thing, Dear Esther and Gone Home didn't fail to be popular. You can be financially and/or critically successful and have a significant fan base and still have people complain on the internet. I mean, COD was virtually guaranteed to be the biggest selling game of the year every year for quite some time and it's probably the most widely mocked and despised modern franchise there is. It's not a binary switch.