They do great work as publishers. As a developer...they're kinda meh. I still enjoyed XIII to some extent, but I didn't even finish it and I'll probably never pick it up again to do so.
Like you, I started on the SMT series with Persona 4. Since then I've played (and beaten) P3P, SMT: Strange Journey, Devil Survivor 2, and Persona 2 IS. I'm hoping to be able to get a PS2 again to play the Digital Devil Saga games and Nocturne.
I know that Atlus will always provide me with a quality game and experience. There's never any doubt about it. The same just can't be said about Squeenix.