Nothing wrong with legos. I find em to be a pain in the rear (especially cleanup), and expensive, but if you can deal with those two, have at it. I, personally, participate in cheaper, and usually lazy-friendly hobbies (I.E. Minecraft), but all it is is a hobby. Everyone has one. Most people who don't enjoy your hobby think it's childish and immature (I think that's a guarantee for all hobbies). I play Dungeons and dragons. Everyone I've talked to about it either plays it, or thinks its immature. I used to make model airplanes. Again, everyone who didn't do it thought it was a kid's thing to do. Old video games? you betcha.
In short - it's your hobby. If you enjoy it, don't give a flip if someone think's it's childish.