Here's the straight dope:
Anything that they have given you willingly is technically a gift. It may not have been convenient, but if you didn't steal it, and there was never any mention of repayment, then it was a gift. If you give your Dad money, and he gives you an IOU, that's a loan. You can be as lenient with the terms of repayment as you like (he can take as long as he likes, as long as it's repaid), but the expectation is that the money will be repaid. If your mother is taking money out of your sock drawer without asking you for it and with no intention of paying you back, that's stealing.
If you want to avoid this all together, put your money in the bank except for $40-$50 that you keep in your wallet and on your person. That way, your Mom can't just take it without you knowing, you can loan your Dad some quick cash if he needs it (and asks for it), and when you need to buy things for more than $50 you can do it with your debit card.