I wouldn't... Think of it another way - when your sick do you want to be out and about or home chilaxing?

Just also - your son's cold might not worrying him the slightest but might make serious complications to the next person that catches it...