It's a tough situation and one that, unfortunately, you have to face sooner or later. I think you will feel better once it is all out in the open, everyone knows and then deal with people's reactions/compaints/pettiness when and if it occurs. You might find that when everyone does know, you will have other people defending you to the people who have chosen to be negative.

We had a falling out with mu uncle when my Pop passed away in 2006. We haven't spoken to him in two years and I always feel horrible about it. He was always such a great uncle and we shared EVERY Christmas together as this big, happy family. But the day of planning my grandfather's funeral he raised a hand to my Mum and then stormed out of the house. I haven't seen him since. I haven't spoken to him since. And I won't because I stand by my Mum.

It's a completely different situation but I completely understand how it feels to be in conflict with family