I suppose I'll add my 2 cents to this already jam-packed list.
YES, you should buy a Dreamcast. And I should say right now that Crazy Taxi with a Dreamcast controller is far better than on PS2. Plus its the original console version.
Also want to say that, in my opinion, the Dreamcast D-pad is better suited for the 2-D fighting games such has Marvel Vs. Capcom, than the PS2 push-sensitive D-pad. It just seems more precise, but it's probably just me...
Heres a quick list with a bunch that everyone missed added in:
Sonic Adventure 1 & 2
Sonic Shuffle (Exclusive. Mario Party clone)
Phantasy Star Online Ver.2 (Free Online play set up at www.schtserv.com)
Shenmue (Exclusive)
Shenmue II (European Import. Worth it.)
Soul Calibur (Exclusive)
Power Stone 1 & 2 (Exclusive)
Dynamite Cop (Exclusive)
Ecco the Dolphin (Exclusive)
Seaman (Exclusive)
Space Channel 5 (screw the PS2 port)
Chu Chu Rocket (It's on GBA too but who cares...)
Sega Smash Pack (Collection of Genesis/Saturn/PC games)
Sword of Berserk (Exclusive)
Draconus: Cult of the Wyrm (Exclusive)
Zombie's Revenge (Exclusive)
Record of Lodoss War (Exclusive)
Fatal Fury: Mark of the Wolves (Exclusive)
Project Justice (Exclusive, a.k.a. Rival Schools 2)
Cannon Spike (Exclusive)
Bangai-O (Exclusive)
Skies of Arcadia (Also on Gamecube, but come on, you want authenticity

)
Marvel Vs. Capcom 1 & 2 (Arcade PERFECT ports)
Street Fighter III: Third Strike
Capcom Vs. SNK (Arcade Perfect)
King of Fighters '99 (Arcade Perfect)
Last Blade 1 & 2 (Exclusive)
Jet Grind Radio (Superior to the Xbox remake)
EGG: Elemental Gimmick Gear (Exclusive)
Expendable (Exclusive, Contra/Smash TV style)
Time Stalkers (Exclusive)
Maken X (Exclusive)
Armada (Exclusive. Asteroids/Sinistar gameplay style.)
Bleem!: Metal Gear Solid (Requires MGS PS1 discs. Not as good a method as the PS2's updated PS1 functions, but definitely noteworthy for the Dreamcast.)
That's all the comes to mind and its probably the vast majority of the exclusive titles. Hope this helps.