Ok, excuse me while I LOL here, but if all he has is a note that says "I yell too much", then he has absolutely nothing. Every parent in the history of the world has yelled at their kids. For some parents, "too much" could be once a month. They have much bigger fish to fry - and he knows it. Don't let that stress you.

Your anxiety is being properly managed, and has never caused anyone with medical experience cause for concern in regards to you being the primary care giver of your children. I think they will take a nice expert opinion over that of a ex-partner (who has ulterior motives) so just get a nice referral regarding the proper management and risk assessment from the treating doctor and keep it aside just in case it raises any questions.

Apart from that, there isnt a chance in hell a judge is going to agree that you have to follow him. If he chooses to leave the immediate area, then that is his damage. If the kids are happy at school and care and have a stable home with their needs met, he will not order major upheval to that. He can certainly TRY what he is trying - it does not mean it is going to be successful - and I am pretty certain based on the way you describe him (sounds just like my ex) he knows it for a fact, and is simply trying to manipulate you to get his own way. He is about to lose control of you and the situation, and when that happens, they start making huge mistakes. So keep records. Don't stoop to his level. Stay strong and play fair. He'll dig his own hole. They always do.