It's a bit odd, and a little creepy that he was watching you work shirtless on your own property... But you were on your own property, outside, in the sun, and doing stuff that would get you warm and sweaty even on a cool day. It doesn't sound like you went out of your way to strut, call attention to yourself, or generally behaving in a lewd way, so it's probably a case of a parent being overprotective and naive about what kids see every day.