I have played every version of Persona 3 and 4, as well as the PSP remakes of 1 and Innocent Sin, and the digital re-release of Eternal Punishment. I have also played both Arena games and Persona Q, and seen Persona 4: The Animation and the first Persona 3 movie. I have also dabbled in the wider Shin Megami Tensei metaseries, and found games like Digital Devil Saga and Devil Survivor to be really enjoyable.
I think it's safe to say that I adore this series, and Persona 4 is actually my single favorite game of all time.
If you're interested in getting into the series, I personally recommend starting with Persona 3 FES, probably the best version of Persona 3, imo. 3 and 4 don't require and previous knowledge of what happened in 1 and 2 to fully understand the plot (there are one or two minor call-backs in the form of name-dropping, but they come and go quickly and are tangential at most). 1 and 2 are still an excellent trilogy of games, but they have not aged particularly well, and could potentially push people away if they start with them. Persona 3 was, for all intents and purposes, effectively a reboot.
You could potentially start with Persona 4 (I recommend Golden if you have a Vita), which made some significant improvements on 3's formula, and you'll probably like it just fine, but it does have a lengthy sequence of scenes that are just an extended call-back to 3, and if you haven't played 3, it will just go way over your head. It can also be a bit difficult, after playing 4, to go back to 3 and not have some of those gameplay improvements.
I don't know how much prior knowledge will be needed for Persona 5, so I'm just going based on the games that are already out here. Anyway, I hope you can enjoy the series as much as I do.