This argument is one of the most infuriatingly arse-backwards reasons for keeping weed illegal, and, irritatingly, it's one of the more wide-spread arguments.
I smoke weed and take other recreational drugs; none of the people I smoke with started with weed. Every single last one of them...