If I'm misunderstanding you OP, sorry. But it sounds to me like there are a couple factors at play here.
For one, it sounds like you maybe picked the wrong school to attend. When I was looking at colleges, I made it a point that I wanted to go to a fairly small school in a decent area with little or no reputation for being a party school. I'm really not big on the whole drinking and partying scene either. I've never found parties particularly fun, and quite frankly, alcohol gets very expensive, very fast. During my four years at HPU I don't recall a single story about a party getting out of hand or police having to teargas student or shootings or stabbings or muggings or rape or anything like that. I wanted a very specific type of atmosphere for the school I attended, so that's what I looked for, that's what I got, and I was more than happy with it.
For another, it sounds like maybe you aren't particularly satisfied with your course of studies. During my Freshman year I really wasn't very sure of what I actually wanted to major in, and ended up switching my major three times before settling on one that I liked. I was pretty much miserable before finding that one. I couldn't bring myself to get out of bed in the morning to go to class, I daydreamed or played video games on my laptop instead of paying attention in class, I half-assed all my work and never studied for tests... needless to say, my grades were unimpressive during that time, which made me feel that much worse, since my parents were footing the bill. Seriously, once I found the right major, pretty much everything changed. I actually (/gasp) liked going to class. I was interested in the course work, and I was now surrounded by like-minded professors and students that I could have good conversations with. Finding the right major made a world of difference for me. So you might want to consider browsing your school's course catelog, and maybe trying out some different majors. You never know, you might find something you really like.
As for your peers... you seem to be generalizing quite a bit about the quality of people around you. During my Freshman year I was assigned a roommate that I hated. I had nothing in common with him and our personalities clashed on everything. With the problems I was having with my classes from having not settled on a major yet and the depression from being away from my home and friends, combined with a roommate from Hell, I was fully willing to get all reclusive. Didn't really bother getting to know anyone because I thought it was just going to be more people like my roommate, and spent most of my time in my room or in the library on my computer, wishing I were at home with the people I actually like to be around. On a suggestion from a friend back home to at least try to be social with people, I decided to attend a school-hosted gaming night in the student center. Ended up meeting two of my future roommates there, who then introduced me to a bunch of their friends, and before I knew it I was part of a big group of people that shared similar interests, had similar personalities, and with similar senses of humor. If I hadn't taken the advice from my friend back home, I probably would have requested a single dorm for my Sophomore year and played recluse for the rest of my time at school, and no doubt wouldn't have had any of the incredibly fond memories of the place that I have now.
After having graduated with my degree, I'm now back home. I've been home for a year now. I have a decent job, but it doesn't make me happy. I find myself pretty much always missing my buddies from HPU and their antics, and the long conversations I'd have with my professors after class that pretty much always challenged what I'd always believed to be hard fact. Though things got off to an incredibly rocky start my Freshman year, I took the effort to make things better for myself, and I look back on my other three years of college as the best three years of my life so far.
Take that for what you will.