Don't steal anyone's slave mistress! That's my rule of thumb (and probably Jefferson's, too).
However, you are changing the discussion from being about "freedom" to being about "damaging welfare". Also, your question is loaded by assuming property rights in the first place. Why not ask, "Do you think coming upon an apple tree and eating the apples damages someone else's welfare?"
I suppose driving cars damages the welfare of others by polluting the air -- but that is equally true whether one is driving one's own car or someone else's. "Stealing" is only possible given property rights. Driving a car that nobody else is using does not necessarily harm anyone (except by polluting).