Well I own a .22 scoped rifle. It's fairly old now, but still in working order. They're great fun if you are sensible. Don't feel too threatened by it, as you seem fairly worried right now. Being nervous with a firearm is not a good thing to do. Like you said, it is still a weapon, albeit a much less powerful one than many. If you are a sensible, level headed person, there's no reason to be worried about it so long as you are careful.
I'd suggest making a target similar to the one I used. Get a sheet of plywood/MDF about an inch thick, then a sheet or 2 of polystyrene the same size as the wood. It only needs to be about a foot square. Then wrap this in thin plastic to hold it together and glue your target on the front. I found that aiming at fences or straight onto solid wood can cause ricochets which aren't really desirable, and you can ruin a perfectly good structure if you don't cover it with something. The target neatly absorbs your pellets and is cheap and replaceable. Then make sure that the target is placed somewhere that has nothing behind it that could be a problem. That includes empty space, as the pellets can travel quite far.
After that, just have fun! Your target should all but the unluckiest of pellets from ricocheting and will provide a safe and effective target.