Of course they shouldn't take priority over gameplay, but they certainly aren't unimportant. Bad graphics can make a good game aggravating, and great graphics can make a bad game bearable.
A game that stands out on the strength of its graphics alone will, of course, be hastily forgotten as it is displaced by the non-stop procession of improvements we are so lucky to be subject to. But if a game's graphics complement an already-strong base of gameplay or narrative, then they can only serve to make an enjoyable experience better.
No one will remember Crysis in a few years once its graphics are quaint by comparison and its staid gunplay is judged on its own merits, but on the other hand I worry what will happen to genuinely strong games that are dwarfed, graphically, even by their contemporaries. Even a great game can prove difficult to enjoy if it feels like you're being poked in the eye by Steve Buscemi, and a lot of games may not get a fair shake by future generations due to how poorly they've aged visually.
All graphics will show their age eventually, but some games definitely age much worse than others, and it's not often obvious why. For example, the more sophisticated graphics of Final Fantasy VI certainly must have humbled the series' previous SNES offerings when it was released. Looking back on them now, Final Fantasies IV and V's appearances hold up remarkably well, while Father Time has really taken his sledgehammer to their successor's more sophisticated visage. And 3-D graphics have a much worse shot at gracefully aging than simple sprites.
I wonder if any developers or game journalists have taken a weather eye at what exactly contributes to the deterioration of games' appearances, and what can make contemporary games, similarly appealing upon their release, weather the years with such greatly differing aptitude? I have a scant few ideas about it, but I've already wandered far enough from the topic at hand.