I just consider Xmas to be the way for people like me who celebrate it as a holiday with no religious affiliation. I know that the Xmas does stem from Christ, but Xmas always just seemed the way to essentially take the christ out of christmas (which many people seem to object to...mainly christians)
Sometimes it slips out of my mouth, but I usually don't say "merry xmas" or "I've got all my xmas shopping done" but sometimes I may just say "xmas is coming"
I don't mind when the guy at Best Buy says Happy Holidays, I don't mind when my neighbor tells me Merry Xmas (if I ever talked to them anyways)
Really it just doesn't matter to me. It doesn't feel any lazier imo, just...different.