It sounds like you are in a very hard situation! i agree with what others have been saying, he says 'do you want him to hate you for do this to him' well does her want you to hate him for makeing you do something you dont want to do.
It sounds to me like he is scared, scared that he has been a crap dad to his two other kids (not his fault but by not being alowed to be there) that how can he possibly be a good dad to this kid!
I think that in time he will come around to it, im sure it will be hard though, and im sure it will set your relationship back, maybe reasure him that you will never do what his ex did to him (take the child away and stop him seeing it) im sure that knowing he has two chidren out there somewhere gives him pain every day and the thought of there being a third in the same situation would be heartbreaking for him.
I really hope you manage to make things work for you, and what ever you decide to do, do it for yourself, not becasue your scared or to please others.
Big hugs to you and amoungst all this heart ache....congratulations!