ah babe, you aren't doing anything wrong. He'll come round, it's truely just a stage. Some kids are just like that. I'm up to my second and he's only just coming around to going to his dad consistently.
My DD was shocking at it to the point where people blamed me, and would walk away with her to prove that I was the one doing the wrong thing.
She is now a bright outgoing, friendly, little girl.
Adults get quite taken aback when forced to face/deal with their fears, so why one earth do we force it on small defenceless children?