We generally would go if its just a cold and DS was mostly his usual self.
How bad is the cold, is it something you'd keep him out of kindy for if he was older? (At DS' daycare they can go if its just a cold and theres no temp, as long as they aren't totally miserable)
How upset is your DS? Can you call anyone from playgroup and see what they think?
If they other parents are ok with it i'd go an stay a short time.