I agree too. I don't think it should be you who has to bear the burden of guilt as to whether they will have to replace you with a casual employee or not. I don't understand him anyway, as many people are employed on 12 month fixed term contracts to cover maternity leave. This is certainly not the big issue he's making it out to be.
Furthermore, it's very wrong and very unprofessional for him to expect you to make a decision about whether you intend to return full-time at this stage, and is certainly over-stepping his boundaries by questioning if you'd be able to cope.
He is legally obliged to hold on to your job for you. You are legally obliged to give him 4 weeks notice if you don't return. Anything outside of this is politics that, frankly, he shouldn't be playing. Your rights are set in stone in legislation. Remember this, as it is there to protect your rights, the rights of your family as well as those of your emploer.
I personally would be even more resolute in taking the 12 months you are entitled to.
Bookmarks