I was in the same boat as you with my DS1. I really should have held him back but he was begging to start school. He struggled the entire way and ended up repeating Yr 5 where he then started to fit in with the others.
I agree to seeing how he goes. Does the school you're planning to send him to offer introduction days? When my older kids started school they would go for 1/2 a day once a week to get them prepared for school and for the teachers to see where they fit.