In relation to sleeping on his side, you could put him in a bubbaroo joey pouch (you can buy them from baby stores like Baby Bunting), they're like a wrap but it's loose so he can move his arms around but can't get his hands up to his mouth to pull the dummy out. It's been a savior for me in transitioning DD from being wrapped tightly.

You could also try the cherry style dummies, they don't tend to fall out of an open mouth as easily as the orthodontic ones? Good luck!