I had probs feeding both my girls and neither were exclusively BF so I know exactly what you mean about guilt/grief.

The constant head battle of wanting to express more but not finding the time really is exhausting. I found it interfered with my entire day as it didn't matter what I was doing - housework/cuddling/playing/cooking with DD1, I felt like I SHOULD be using that time to express. I think it's worth looking into whether shorter but more regular expresses to build supply (rather than building an EBM supply if that makes sense) would be do-able. FWIW when I was relactating, I found it OK to express or attach DD2 on an hourly basis for 5 mins here and there - much easier than expressing for 20 mins every few hours.

The only thing I can really tell you is that the grief eases with time but right now you're in the thick of it. I found that I was much more at ease with my decisions at around the six month mark when you can see that they're thriving.