Hmm... rather than saying where I draw a line on nerdy things, for me it has more to do with the kind of people I want to associate with. I mean, most nerds are well-adjusted enough to pass for normal folks in most situations, but I've known some nerds that are so nerdy, maladjusted, or just plain weird that I'm surprised they can interact with people at all.
It's alright to be into nerdy things, but there are some people out there who let their hobbies and interests take control of their whole lives to the point where they need to give it a rest. This happens a lot with Anime nerds; they start mingling Japanese culture references into their day-to-day conversation so often that you just want to slap them and say "YOU'RE NOT JAPANESE! YOU'RE , YOU DUMBSHIT! STOP ACTING LIKE SUCH A NARUTARD!" There's this vlogger chick on YouTube that does this kind of thing, wearing cat-ears and all that, but people let her get away with it because of her tits. Is that an offensive and sexist statement? Yes, but the Internet is an offensive and sexist place, and even in real life you all KNOW that tits let you get away with things much worse than obnoxious culture-crossing.
And it's not just culture nerds either. This applies to science nerds too (perhaps even more so). You know the kind; those guys who are super-smart and are basically walking libraries of mathematical and scientific knowledge, and yet never wear anything besides green cotton shorts and math competition t-shirts. Those guys you just want to shake and say, "IF YOU'RE SO FUCKING SMART, WHY DON'T YOU REALIZE THAT THERE ARE BENEFITS TO DRESSING YOURSELF PROPERLY AND COMBING YOUR HAIR!" I mean God damn! I had a roommate who was like this a while ago. Thankfully, he at least dressed in a shirt and tie every day, but he was still sucking his thumb at 25 years old! It was kind of a different situation in his case because when he grew up everyone thought he was mentally retarded (even though he wasn't), so he didn't have the same kind of social interactions that most of us had, but still, sometimes you have to put the book knowledge aside for a while and work on other areas of your personal development.
And, of course, the worst part is that those people who are THAT nerdy about something tend to be clingy (Except for the ones with tits; unfortunately. They have enough groupies to deal with as it is). If you show the slightest bit of interest in them or what they're doing, suddenly they assume you're their best friend and will dog your every step no matter how hard you try to avoid them. And if you put your foot down and tell them they need to back off, they swing to the opposite extreme and assume you hate them and never want to talk to them again.
And last, but most definitely not least, are the asshole nerds. The people who take their nerd thing waaaaaaay too seriously. Those guys who know everything there is to know about Magic: The Gathering, or D&D, or World of Warcraft, or are top-ranked Halo players, or can name all the characters from every single Gundam series, and they get a huge superiority complex about it. Then when you start to interact with them at a novice-level or intermediate-level, and you say something wrong or make a simple mistake, they jump all over you like you're some kind of idiot.
So basically, I'll try anything no matter how nerdy, but the degree to which I'll get into it is limited to how much I enjoy it, how expensive it is, and, most of all, how deep into it I can get before I start running into those kinds of people. I'm sorry, I know this sounds terrible, but like OP said, everyone draws the line somewhere. For me, it's not the nerd-thing, but the people surrounding it that make the most difference.
In hindsight, this is probably why non-nerds rarely get into nerdy things. They see these kinds of people and think, "Damn, I don't want to be like that guy!", and it saps all their interest. A lot of the time they're just being shallow; most nerds I know have great personalities but are lacking in social advantages like being good-looking or athletic. Still, when I see people that are like the people I've described above and are into something I've got a passing interest in, the same thing happens to me.
