in my opinion, of course you can take him out at night if he is only going to be in the cold air for a couple of minutes at a time!!! it sounds crazy to me that people would keep themselves locked away at home just so bub doesn't get a bit cold.

i just used to get him out of his carseat and hold him against my chest and tuck a thick flannelette blanket around him and under my arms so the cold air couldn't get to him.
We went to a 21st party when he was 6 weeks old which was held in someones back yard, just made sure he was rugged up and he was either nursed by someone or sleeping in his stroller and he was fine and happy.

There are so many better things to feel guilty about, please don't feel guilty about taking your baby out of the house