my x blamed me for his cheating, even though he couldn't keep his **** in his pants the whole time we were together, even when dating.

trust is something that if it is broken, is the end of the relationship and I will walk away

but you need to do what is best for you and your family, in the end what happens in your family is your own business and you will have to live with whatever decisions you make