I'd say any more than 4 hours is too far to go for much, excepting trips/overnight visits. Anything over a couple hours is probably far away, too, for me.
I live on the Canadian prairies, so it's about an hour and a half to the US border, and then an hour past that to Grand Forks/Fargo, and that trip is made almost weekly by many people, so that's not too far. I"ve got family in a town about an hour and a quarter north of me, which isn't bad at all.
Keep in mind, these times I'm giving are considered far drives in good weather. If it's winter, anything more than 2 hours is too far. You basically have to tack on 15-25%+ more time driving in the winter if you're not in the city.
Some people don't realize how big Canada is - I live in basically the middle of the country (the actual longitudinal centre of Canada is about 30km from here) in Winnipeg. Look at that on a map and realize we're about 100km away from the US border. For me to get from here to the foothills of the Rocky Mountains in Alberta? About 16 hours driving, with stops for bathroom and food. To get through the mountains to Vancouver? Probably closer to 26 hours, straight driving, only stopping for food and bathrooms. Maybe faster, but not much.