I don't see an issue if you are only going to be outside from the car to the house etc, for a short period.

I wouldn't do it wil DS very often now, but thas because he goes to bed at 7:30 and doesn't tend to sleep well at other people's houses. Up until he was about 6 months old we used to take him to my Aunty and Uncles house once a week, unless it was raining or he was unwell.