I used to think the exact same way about Facebook, but, as it seems you've found out, it's the only way to get/stay in touch with some people. (As I myself found out).
So I gave into peer pressure and got an account.
How's it impacted me? Not too much - I check it when I log on for 5 minutes. That's the extent of my dealings with it since I joined.
I remember I got a friend invite from one person from my old school that I, with no slight exaggeration, hated unconditionally. A guy who'd made my life hell for four years or so. At first, it did upset me; then I thought... why? I clicked ignore, and had a laugh about it... and I haven't heard from him since. Remember that you don't have to accept friend requests, or even put down any info whatsoever.
As for your friends, I guess it depends on how close you are? Is it really worth losing these friends over something like this? Facebook definitely has its downsides, but it does have its uses. It makes organising social gatherings/parties a hell of a lot easier. Its made keeping in touch with friends of mine living on the other side of the country easier as well. Cheaper than constantly texting or calling them (I'm poor, you see).
Eitherway, just remember that you don't have to be on it 24/7.