Honestly, I think if you feel that way and you don't need to leave him then don't. It's ok to feel that way - in fact it's natural and normal. It certainly doesn't make you a crazy lady.