To repeat the mantra of a dear friend of mine, the more advanced we become, the more limitations are put into place.
Yes, many old games were awesome, and really deep, and beautiful, but would they work today? I personally loved the dialogue in Fallout 1, or Arcanum, or whatever.
But today, would it work, technically? Players would demand talking heads for every important character, and voice acting for almost all of the dialogue. The amount of dialogue in any of those games are staggering. Recording it all, and animating the dialogue today, even creating the models for all the npcs in game (unless you use the Half-life way, with about 10 models!) would take so long most people wouldn't even dream of trying. It's not doable anymore. Sad perhaps, but true.
I mean, look at it. If you want to create a scene, a village, of 300 people, 50 of them have their own personality, and each of them have opinions on all the quests you do, and there's five ways to solve each quests, of which there are hundreds, how would you do it? If we look at the simplest engine ever made, a text-based game. That takes maybe a month of concentrated work (the village, not all the quests). In that month, you have created a situation where you can fight the whole village, you can save it from a dragon, you can kill the dragon and the village, leave the dragon to kill the village, or tame it, and make it the guardian of the village.
If you try the same in a 2D engine based on sprites, you can still make it pretty easily. It takes maybe a year instead though, but you can reuse a lot of the graphics.
Try to make it in full 3D today though, with full audio recordings of all the dialogue, with custom models for all the npcs, with several different solutions for the quest, and several models of every npc and building (burned down, broken, killed, alive, rebuilt) and make the different versions of the dragon, the newly built dragon lair, and you're looking at 5 years of concentrated work.
Sure, we've got amazing graphics these days, technical graphics, and we've got really pretty audio effects, and the immersion can be good (it's not, since most games these days suck though). But as we gain this, we lose a lot of other stuff. As we get more advanced engines, they also become more limited, and harder to work with. Sure, what they can do they do well, but they don't really do well with new stuff.
Look at the size too. The first example is 100 kb, the second example is maybe 200 mb and the third example is 5 gb.
Some people, mostly people who have gamed for a long time, can accept the old way of making games, and some indie developers still make games that way. They can be found. But the mainstream, the regular gamer, while nostalgic, couldn't accept a game like that these days. It's again sad, in my opinon, but there's not much we can do about it.
Well, except try to make such games yourself, but let me tell you, and this is from experience, it's BLOODY hard work. It's hard enough to make a simple, linear game today without a huge budget and a big team, trying to make something interactive and cool and stuff, that takes effort. A lot of it.
However, when someone puts Half-life 2 forward as an example of an intelligent game, I am inclined to agree about the decline in gaming. I don't think I've ever played a more shallow, boring, linear game.
) 
