there are lots of situations in which people don't know the address of the other parent (changeover occuring at swap over points) - i don't think you would be able to deny him based on that

i agree with Lulu - he doesn't deserve to see your beautiful little girl BUT, i don't think you'd be able to deny him based on the address situation.

if you want to be sure, contact the family relationships advice line and get it clarified through them or an associated parenting advisor