Perhaps when your partner gets home, walk out the door and leave him with everything to cope with.

Honestly, he sounds like a bigger problem than your little one.

I agree with Mad B. Try chatting with your maternal child nurse. Or, if you are really worried that something is not right, see a pediatrician.