As a rule of thumb, my fav PS2 games are the ones that have both stayed with me one way or another, and also the ones I've replayed out of sheer need.
Okami
In a world of dreary doomsday scenarios and diabolux ex machina downer endings I find this game to be one of the most uplifting, cheery experiences on the PS2 and so many other consoles. It's one of the few games that's stayed with me long after I played it. The Celestial Brush mechanic is fun and nicely worked in both combat and puzzle solvin, the graphics are amazing, not just for 2006 but the art that's gone into them, and the music is, no other words for it, triumphant.
Shadow of the Colossus
This was one of the very first games I played on the PS2 and it set up a visual and artistic standard that was hardly ever topped by anything that came after. I've always admired how they accomplished so much with so little elements - the sound design, the bloom effects, the fact that you wanted to explore the whole landscape to drink in the visuals even though there were no NPCs to meet or enemies to fight. Atmosphere at its best.
Killer7
The whole thing kept me on my toes for as long as it lasted. I was creeped out at the beginning, then I was properly amused, then I fell for the weirdo comedy act, then I was touched by the Hotel Union reveal. This game goes for a wide array of emotions when it comes to storytelling. As is the case with how it tells it: rail shooting/character upgrading/manga 'n anime segments/cel-shaded noir/spyspeaking... the list goes on and I love every moment of it.
Dragon Quest VIII: Journey of the Cursed King
It's a pretty conventional story and I always thought the game suffered from its blank, nameless protagonist, who was OK to tag along so long as the story wasn't about him (which it wasn't, at least until the ending/secret finale). I also always thought the fully 3D, cel-shaded enviroment sans loading screens was the most gorgeous thing you could go on for free roaming on the PS2. I love the character design by Akira Toriyama, how every single NPC gets a distinctive look. Also, same as Okami, the game goes for cheery adventuring rather than its more popular, grim counterpart.
Silent Hill 2
I may be a little biased towards it. Not because everybody keeps pointing out how good the story is, and how it features in every Top 10 list. In truth, it's the first SH game I ever played. I was introduced to the whole monster-manifestations-extrapolated-from-the-protagonist's-guilty-subconscious thing through SH2. Having played through all other PS2-era SHs, I never found any other game that approached character development, storytelling and terror so subtly and properly. It's become a milestone to me.
Resident Evil 4
Well yeah. RE4 came out in 2005 and I'm starting to think that was the best year for the PS2: GTA San Andreas, God of War II, Kingdom Hearts II, Shadow of the Colossus, Killer7... it arrived at a peak of excellence. Much as I love good old fixed camera angles and proper zombies putting the horror in survival horror (and the survival, too), this game is just too much fun. You can tell how RIGHT it was with the whole camera-over-the-shoulder thing getting ripped off by every 3rd person shooter that came ever after. And sidestepping the Umbrella arc was a wise, wise choice as well.
Mercenaries: Playground of Destruction
Also came out in 2005, and it's my personal pick of the sandbox bunch. Inasmuch as I love and played through the whole GTA III era, Mercenaries reduced story to a minimum and went for absolute, fun and indulgent gameplay. Rarely have I played though a game that gave you so many options as to how to carry out a mission and congratulated you regardless of your method of choice. Tracking down and bringing in cards alive was too much fun. I loved calling in extractions and air strikes, falling in and out of favor with the warring factions... Word to the wise though - the Ace of Spades was too difficult.
Obviously I can go on and on, so long story short: Odin Sphere (gorgeous hand-drawn 2D action-RPG), Tales of the Abyss (for Luke's amazing and believable character arc), Prince of Persia: Sands of Time (gave us the rewind mechanic), Kingdom Hearts (if anything because it got the FF/Disney mix right), etc, etc, etc...