The specs you quote should be able to run the game fluidly, but maybe not at highest settings... what resolution are you running it at? What setting? Is your drive defragmented (yes, outdoor terrain is paged into memory from disk and a fragmented drive will hurt).
I have a slightly better GPU than you (of nVidia type) and can run 1920x1080 at Extreme settings, but not without hiccups. Dropping the resolution or changing a step down from Extreme helps.
Incidentally, CryEngine is a much newer engine than Gamebryo and in practice consumes less memory, leaks less, fragments memory less (something Gamebryo is notorious for, especially with games that drop and load a lot of resources), and performs better, mostly due to needing much less legacy code. OTOH, Gamebryo has a lot more features and includes a lot more third party utilities (like Havok physics and SpeedTree). At the time Bethesda chose it (almost a decade ago - circa 2002), it was probably the best choice for an RPG, but I don't think the engine itself has ever had a full redesign and has always evolved (it started as NetImmerse).