It usually depends on the context for me. I usually say passed away if it's around strangers or mixed company, but if I'm with people I know or it just seems appropriate I'll say they died. Recency has a bit to do with it, too. If it's more recent I'll say passed away, but if they've been dead a while I'll say died or dead.
It's just all about appropriateness and respect, really. Saying someone is "dead" around those who knew or might have known them almost trivializes it in a way, and makes it sound gossipy or disrespectful.