Dont feel like you have to let someone babysit your DS, its your choice on who and when you decide to leave your DS with. When you are ready and feel like its time you have a break then leave your DS with who you want to. It may do yourself and DH good to spend some time together on your own. Sometimes men feel abit left out when your busy with a bub all the time. Its hard too leaving your bub when still BF'ing. Remember the first time you leave your bub with someone/someone new is always hard.
You will know when the time is right. hugs