Yeah, it really depends on what you're looking for:
Like adventure games and playing strategic board games with players across the globe? Further exploit your iPod Touch, so long as it's somewhat up to date. @Rack is absolutely spot-on with his recommendations.
Like gaming the way it used to be, colorful and ungodly difficult on single-player campaigns, without studios having to throw all their creative money into pushing more polygons? The 3DS, or as @Retrograde said, the DS Lite, as the GBA library is freaking amazing (WarioWare: Twisted, Advance Wars 2, Final Fantasy Tactics Advance, Metroid Fusion, etc). I couldn't say quite the same thing about the DS library, but there are some gems (Kirby: Canvas Curse, Rhythm Heaven, Pokemon *anything*).
Get a 3DS XL instead of a DS Lite if:
*you really like Atlus' hardcore series (Etrian Odyssey, Shin Megami Tensei)
*you're an Intelligent Systems fan (Fire Emblem, Advance Wars, Pushmo/Crashmo)
*you're a Mario Kart fan (MK7!)
*you're fine with iterative refinement of current IP instead of new or more obscure franchises (Nintendo)
Vita, though, that's a tough sell until the PS4 is out. It looks like the remote play on that will be amazing, but the big featured games out there either are trying too hard to duplicate a home console experience (Uncharted, AC3) or are also available on PS3 (Sound Shapes). As a media player, yeah, it's great, but so is an iPod Touch, so I'm wondering if you need that.