Here's my favorites, I really recommend them if you missed any..
-- PS1 --
Metal Gear Solid - one of the greatest games of all time and most vote it the best PS1 game.
Syphon Filter 2 - Really good 3rd person stealth action game.
Tomba 1 and 2 - Rare hard to find games but good funky plat former with RPG elements.
Suikoden II - Rare and expensive but one of the best RPGs of all time.
Final Fantasy IX - I like this one the best but they're all pretty good 7 and higher.
Jersey Devil - Underrated platformer puzzle type game.
Medievil 1 and 2 - Really good, make sure you check these out.
Heart of Darkness - Great puzzle game
Fisherman's Bait - Fun fishing game
Dino Crisis - Outdated controls but interesting story
Resident Evil 1, 2, 3 - Outdated controls but interesting gameplay
Ninja Shadow of Darkness - Old school style ninja platforming game
Brave Fencer Musashi - Fun action RPG
Vagrant Story - Good dungeon crawler
Jackie Chan Stuntmaster - Fun 2D platformer
Silent Hill - Awesome horror game
Rayman - Fun platformer
Klonoa - Fun platformer
Most of these games on this list are on PSN but the ones that aren't are gems very worth getting.
-- PS2 --
Resident Evil 4
Half-Life
Silent Hill 2 - Great sequel
Metal Gear Solid 2 - Great sequel
Devil May Cry - part 1 is the only good one on this console
GTA 3, San Andreas - Both these are great but graphics will be awefully outdated nowadays.
Ico, Shadow of Colossus - Super games by a unique developer
Fatal Frame 1, 2 ,3 - Great survival horror games
Final Fantasy X - If you really dig RPGs
God of War II
Jak and Daxter Precursor Legacy
Kingdom Hearts 1, 2 - Good RPGs
Onimusha 1, 2, 3
SSX Tricky
Sly Cooper 1, 2 ,3
Disaster Report
Herdy Gerdy
Okami
Rayman 2
Ghosthunter
Klonoa 2
PS2 is loaded with good games, too many to list but those are my favorites.