Oh wow. Okay, if it were my DH, after the issues we have had in the past (no violence but anger issues, addiction etc) yes, it would be the last straw. Hitting children is not on here, I'd be devastated if anyone hit DD so hard you could hear it from the next room.

You say you're surprised he did it. Are you really though, deep down? If he has all these self control issues (getting angry at a toddler for doing normal toddler things all the time is a sign he can't manage his emotions properly, namely frustration), and that he's irritable, shouts etc, it sounds like this is an unfortunate progression of his anger going unchecked. So for your situation...I'm not sure. I would want a pretty good apology from him, both to me and the kids. I'd be looking out for him to actually admit he was an idiot, admit WHY his behaviour was wrong and stupid, and HOW he could avoid ever doing that again. If he couldn't identify those things, and just came up with excuses, justified himself or said he wouldn't do it again with nothing else, yeah, he's be out the door. Maybe not forever, but excuses, justifying himself, even just being flippant or irritated when you bring it up are big warning signs that he hasn't learned from it and will do it again. The only appropriate reaction after doing something like that should be remorse, contrition and shame. Not anger. So yeah. That's my answer. I'd want him sorting out his anger issues with counselling once and for all too, because that's not just going to go away on its own, and as the kids grow they're going to become aware of that, and if he's not going to step up and make sure they never have to see that again, then you need to, for their sake.