Your DH is acting like a selfish child.
You have done the right thing. Unless there's a threat to your child from knowing or contacting their father, your DH has no right to stop them having a relationship.
He's manipulating you and being emotionally abusive.
I hope he starts being reasonable soon.