TL/DR: restrictive parents, my own interest, ostracism by peers, natural intelligence.
I don't like sports because my folks don't like sports. I believe fandom is usually inherited. Not liking sports meant I had to like *something*. Nerdy stuff was as good as anything else.
I got into books, video games, and comic books because my mother was paranoid and excessively restrictive. I wasn't allowed out of sight of my folks' house until I was thirteen? All the neighborhood kids could go wherever they wanted, which meant they were always far away. Without much in the way of real-life friends or outdoor activities, I turned to worlds of imagination.
I always sucked at sports, which gets you pigeonholed as geek, fag, lamer, whatever the word of the moment is. Coming up through school, whatever the best athlete says about you is as good as true. This further encouraged me to keep my head down and not talk, living inside my own mind.
I consistently score 98th-99th percentile on intelligence tests. I waltzed through school without effort and everyone knew it. I won't say I was terribly interested in the coursework but I read a great deal outside the textbooks. I actually enjoy knowing how fast light travels in a vacuum, how methane molecules are structured, things like that. If you aren't already considered a nerd, knowing that kind of thing will seal the deal.