From about 3 weeks old my DS got upset if anyone but me held him. DH could get maybe 5 minutes but then DS would be crying for me. It was hard going. I think he was around 15 months old before DH could take him away for an hour or so. DS remained cautious around strangers for his toddlerhood - not so much talking but did not like to be held/touched by strangers (including grandparents).

He did outgrow it eventually and is now happy to hug the gparents and tolerates random hugs from other people. He has an anxious personality. I think he was pretty extreme though, lesser degrees of separation anxiety are completely normal.