Whoa, hey. Yes, yes it does. Have you heard what royal family lineages were like, back in the day? Inbreeding in those populations caused some serious issues. Notice how purebred animals have strong and specific medical issues, like torsion in Akitas and hip dysplasia in...I think German Shepherds. This is because of inbreeding.
No, your genes don't magically screw up because you're having sex with a close family member, but who knows what little genetic problems are lurking, and even with technology, chances are you'd never see them until they popped out in your kids.
My opinion is, really now, don't do that with your family. There are so many other people out there. Just...don't even. I'd certainly not do it myself.