People have been trying to answer that question for 30 years. The problem is, everyone has a different opinion. For example: I don't consider most JRPGs to be actual RPGs. A lot of people would probably like to see me burn in hell for that, but so be it. So yeah, no way this debate is being ended here or anywhere.
Personally, I think the most important aspect is the ability to define your character. This means his abilities and skills, but also his personality, and how these things interact with each other.
For example: A character who has very high persuasion skill, and uses this skill to trick people in nasty ways, is pretty much a slick, nasty son of a *****. It wouldn't be possible to play as this kind of character in a game that doesn't give you control over both skills and personality, unless it was per-defined, and therefore, it isn't a TRUE RPG.
I realize this definition is extremely limiting, and no, I don't use it consequently, since it's incredibly rare and can often be found in differing degrees.
I don't consider Mass Effect to be much of an RPG for example: Yes, you have control over your character, though I find it painfully limited and completely predictable, without any real nuances, and you can choose from 6 different classes. BUT those classes have no bearing on the plot or even your interaction with other characters, and therefore they do not contribute to the RPG elements.
Be aware: I am bashing some of my own favorites, like the Baldur's Gate Series, or Gothic I & II, as they don't really fulfill these criteria either, but for my personal taste, the "perfect" RPG should contain these elements. Games which I think have done this well are: Planescape: Torment, Arcanum, Fallout 1 + 2 + New Vegas (No, not 3)