Hmmm, I'm not happy to read that he gets 'mad' at you for bringing it up, chicky.
I'd like to see you go to a counselling session together - sounds like you need a mediator so that he listens to what you're saying, he'll have to mirror it and if he gets the message screwed up again, the counsellor can help him understand what you're saying, not how he feels about what he thinks you're saying (he can talk about that after you've had your turn!).
Does that make sense? It's causing you so much hurt that it's only fair he do this for you, just so that you don't feel the door close on feelings that are very important to you and do put an end to this damn 'getting mad' crap - that's unwarranted and makes things worse, and quite frankly, I'd like to find him and slap him upside the head for copping out and just shutting you down like that.