Simone,
You really need to stand up for your self & say no to this man. Don't let himtreat you this way. Even if he thinks its all in your head, surly he could just hummor you for a bit to put your mind at ease!