Well, since you already stated you will be getting Persona 4 Golden, I won't waste my time recommending it further.
I know you said you wanted the physical cartridge, but I would still definitely recommend PS+ if you don't have it already, as long as you are willing to play one game at a time. I've only been using a 4 GB card, but I sank a ton of hours into both Gravity Rush and Uncharted: Golden Abyss (both free with PS+ and seem to be there to stay). Currently Soul Sacrifice (which will be the next game I load up), Oddworld: Strangers Wrath, Wipeout 2048, and Sonic & AllStars Racing Transformed are free as well at the moment, though Soul Sacrifice will be gone next Tuesday. Titles swap in and out every week or so, so you can always have something new to play. Plus, now you can re-download titles that you "purchased" through PS+, so even if you don't have the space on your memory card at the time, you can still grab the game while it's free and download it later!
If you are a fan of the Assassin's Creed series, Liberation was pretty enjoyable, spicing things up by using a outfit mechanic to allow you to blend in with different groups, but limited your capabilities depending on what you wore. I've heard great things about Tearaway and Killzone Mercenary (I have Killzone, but haven't played it yet since I'm trying to platinum Persona 4 first).
If you like puzzles, Zero Escape: Virtue's Last Reward is awesome, but it's half visual novel, half gameplay, so I can see that it might not be for everybody. Also, it is a continuation of 9 Hours, 9 Persons, 9 Doors which was a Nintendo DS release and I feel is necessary to play first (or at least read a wiki) in order to recognize some of the plot points of VLR.
While there do not seem to be a lot of Vita games when you browse in a store, there really are a good number of titles at this point if you are willing to order online. Have fun and enjoy your Vita. I've been using mine daily for the past 7 months or so.