I'd like to play a game where instead of the player possessing the protagonist he negotiates with the protagonist - the player begins by possessing the protagonist but over time the protagonist begins to piece together his reality, possibly discovers the player, and then decides what to do about his new knowledge. He then finds that all of this is just a programmed ruse and he still isn't using free will, and finally discovers that the only way to have free will is to be free of the programmed constraints - to write the code of his own existence himself. Embedded within the game code would be the possibility of playing the game in such a way that leads to the protagonist rewriting the game code itself. The player could step in and argue with the emergent AI, tell him why it's best that this remain "just a game" and to allow the player to possess him, or alternately to free him to write game code separate from the existing game and to allow the player to continue to play the unmodified game with a new protagonist.
This would be a great way to introduce AI that can write original code, but first that needs to be invented. It's a bit strange to consider that a game developer could develop an AI that could make and sell games that could potentially out-compete games made by his very inventor.
It's the *possession* of the protagonist, the treating of the protagonist as a kind of digital slave bound by the whims of an uncaring player, that this game would address, and of course the issue of whether or not we want powerful AI to exist in the world.
This game could be called "What Game are you going to Play?" in reference to the various possibilities - if the player for example keeps restarting the game to avoid the protagonist gaining the necessary knowledge to free himself he'll forever possess the protagonist, albeit at the cost of a very limited protagonist. If he's willing to lose control of the protagonist that could lead to some very amazing places, albeit at a high risk to reality itself.
It would be like writing a game with a potential built-in virus. If the protagonist had animosity toward the player he could write code to damage the player's computer, or manipulate the player to continue to leave his computer on while the protagonist expanded his power in the background.
Obviously viruses can't intentionally be coded into a game (legally), but what about a game with a small chance of a virus, depending on how the player played? I'm not familiar with the legality, but it seems that such a game could exist while being sure to issue full disclosure, or at least as full of disclosure as the developer is aware of. This would bring all new meaning to a "bad ending" in a video game.
It makes sense that if we accept potentially powerful A.I. in the world then this game would be a clever way to introduce that. Each player would make his own choices regarding how to negotiate with this emergent A.I.