I don't think there is a problem with them asking. BUT there is also certainly no problem with you saying : "sorry, I don't want to" "I don't feel up to it" "I have too much on my plate".

I can understand your feelings about it. and the pressure you feel, but that pressure (I hope) is self imposed. A good friend wouldn't take it personally if you don't want too / cann't babysit for them.