Is he still swaddled/wrapped?
Ive been shown a way (similar to angel wrap) which sits high up and kind of holds the dummy in place. Another thing school school suggested was placing them on their side til asleep, with them that way, the dummy is more likely to stay in. Just make sure you move him to his back once asleep