Many people say exploration. I suppose that is true. I'd like to inflect exploration though. Because, for me, exploration by itself isn't enough. I think that in the best open world games, the location itself is also a character. Exploring is not just an end to itself, but a way to learn more about the world you are in, and ideally will inflect the central gameplay.
I think Morrowind was the best at this. All the exploring I did in parts of the world revealed to me complex culture and politics of the Dunmer society in Morrowind, it's relationship to Dunmer on the mainland, and its relationship to the Empire. I learned about religious conflict, cultural conflict, tradition and innovation. All that I learned made my journey towards Nerevarine much more nuanced, rich, and complicated.
Quite a few people said they didn't like Oblivion as much as Skyrim and Bethesda said they learned their lesson--more weirdness to explore! But I think that was the wrong lesson. What made Morrowind so great as an open world, was the real complexity of society they achieved.
Then I think about another great open world: LA Noir and True Crime: Los Angeles. Both did such a wonderful job recreating Los Angeles, the feel of it's different places and locations. Places I know really well! It was wonderful to catch the feel of something so well.
And also, I think for me, the open world means that the game can focus on the small as well as the big. What I mean by that is this: In a linear game, the story is usually about how great you are as the main character. Everything revolves around you. It is quite solipsistic and self-centered. But in an open world game, you can come upon communities that aren't caught up in your messiah, save-the-world mission, that just need fresh drinking water or to resolve a conflict between neighbors. Without small things, the world can feel hollow.
I love a good open world.
A bad open world on the other hand...is just bad.
But then, bad linear games are bad, too.