Do some Googling about third hand smoke and babies. This will provide you with evidence to give your DP about the effects of the chemicals in cigarettes on babies. It is not just when someone is smoking around the baby but the residue left on furniture, clothes and skin (eg hands holding your baby) hours even days after and that newborns are susceptible to low levels of these toxins.

Maybe also reassure him it's not that you don't want your baby to have a relationship with his Mum it's just the smoking that it the issue.

Tricky situation good luck!