The only thing I can suggest is talking through the reason for his anxiety in a logical way. I got that from my psychiatrist, because I've done it my entire life. I have a very clear memory of going home in the car from a party at 7 years old, thinking back over what had happened that night, and telling myself off and getting embarrassed at things I perceived as 'wrong' behaviour. It was nothing actually wrong, just stuff I thought was dorky. So I do believe that his behaviour could very well continue into adulthood, unless he learns to think it through logically now.

For example -

What is making him anxious? His red/green shirt.
Why is it making him anxious? Because it's not the same as the other kids.
Are they teasing him because he's different? No. (my assumption, or the teacher would've told you)
Would he tease someone if they wore a black shirt, because they looked different? No.
So what is there to be anxious about? Nothing.

I'm not saying it's gonna work straight off, but maybe introducing that line of thinking whenever you notice he's getting upset will help him stop the anxiety. I did it myself yesterday, when my car broke down. I ended up sitting there for 3 hours, inconveniencing acquaintances and abandoning my uncle to walk home in the rain. Not that I could do anything about it - so there was no point getting upset and anxious. So I didn't.

It's not easy, but hopefully learning young will make all the difference for him. Also, you're an awesome Mum for picking it up so early and being proactive about it - I think some people are just more prone to anxiety, and need to learn the coping skills early, and you're totally doing that for him