It sounds like he is trying to lay a guilt trip on you, and you are right, he is trying to make out that you are the one that is being silly about this! You are not!! You are doing what is right for you and your children and your futures! Don't let him get to you.
As for sitting it the front of his ute, well, that is totally your call! You are the parents and you make the rules, he doesn't get to make them for your kids, and if he cant stick to them, then he doesn't get to take them in the car! Plain and simple!!
Stick to your guns babe, and whenever he annoys you, come and vent in here, we are always here for you xxx