For me, it's kind of relative. Once you get good at a game, it starts to become easy. That's a no-brainer. The more times you play it, the better you get at it. But there's also difficulty in the sense of how much you need to learn in order to overcome the difficulty.
For instance, Touhou 6 is a ***hard*** game. Even on normal mode it gets nuts. But that's only because you have so much to dodge and you have one hit box to focus on. But yet, there's not much to really learn. It all comes down to a matter of survival while pinning your finger to the 'fire' key. It takes time, many trials, and some frustration. And that's also another reason why it's considered difficult. But in time playing the game becomes second nature. And despite every failure I've had while failing it, I can say that the game is balanced enough.
And then when you play Touhou 7, the game feels run of the mill, and doesn't seem hard at all. But that's only because you've figured out already what needs to be learned. And most likely, you've learned what needs to be learned. There isn't much that needs to be learned for games like that, it's just that it takes many trials and practice to overcome it.
Armored Core 3, and its family, on the other hand is hard because there are more items that you need to pay attention to. Stuff like overheating, energy conservation, armor points versus defense points, and energy weapons versus solid weapons.