Here's another perspective:

Even if you have the baby here, and go and visit over there, it is possible that he may bring Court proceedings for visitation with baby whilst you are over there. He may also get Orders which restrain you from coming home.

There is no way in hell I would advise you to go back there. He can come here and visit. Don't do it.

You are NOT responsible for your daughter's relationship with her father. He is. If he wants to step up to it, well and good. If he doesn't - well it's HIS loss, not yours. Your daughter will make her own way in the world, with or without him.

He can come and visit her in Australia whenever he likes. It is much easier for an adult to travel, than a child to travel. If he wants to get to know her, he can. Just make sure it's in Australia and not over there.

Good luck - get some independent legal advice ASAP.