I think you will find it hard & might feel guilty.
Could you return part time for a few months? Or just a few days a week?
Centerlink does have payments, though they are not much if you want a bit more time.
It is totally up to you. It is up to how you are going to feel about the time you are missing with your baby.

I am a person who had kids to raise my kids though. I didn't have them to go to work & let someone else get all the first smiles, laughs, steps. Etc.
I do envy all the women who do go back to work to provide the best for their kids, but emotionally I can't. Casual maybe, not yet though. Full time I might be able to do after 2 years, but not before then.

I am in a very different situation to you though.