I go to the gym regularly.
Muscle development stays with you your whole life. It's also easier to obtain the younger you are.
And there's a lot more to going to the gym than "going to the gym."
The exercises you choose to do can help you strengthen weaknesses, or improve overall strength, endurance, health, cardiovascular, burn excess calories or simply an exaggerated method of stretching your muscles.
You don't just go inside like it's a night club and poof, you're 'that guy'.
Just because your currently skinny because of growth spurts doesn't mean that your current diet and daily exercises will keep the frame you have now. Far from it, you'll absolutely put on weight unless your vegan for some reason.
Plus I know a lot of assholes that go to the gym
Weird, I found that the people that go to the gym are usually the ones who are either there to work out and don't want to be bothered, or don't want to be bothered and work out to gain that 'aura'. And the people who are social are usually the friendliest people I've ever met.