it doesn't matter if a fear is irrational - it is still real.
Maybe you can ask your midwife if she can put you in touch with a grief counsellor.