Did you want to take the contract? If so, I would accept and talk to him about your ideas for how you want to handle you maternity leave and see how he reacts. I'm an optimist but maybe he thought you were upset because you love the job so much and don't want to take time off due to money or something and he was assuming you could still work from home with the mortage broking after bubs is born. Goo dluck