I think we judge ourselves very harshly when we fall short of the ideals we set ourselves as parents. But sometimes life conspires against us.

It sounds to me like the current arrangement just isn't working for you guys, and some sort of change is necessary.
Do you have to wean completely? Could you keep bfing when it suits you and use formula at other times? That could at least give you a break from this exacting schedule and allow you to reassess the situation after having a bit of a breather.

There's no place for guilt in decisions you take in the best interests of your family.