I just second what JwithJ said - try to make sure the material is spread out as wide as possible, especially across your back and shoulders... And tucking the edge of one of the shoulder pieces at the front around his head a little usually helps stabilise it. I just made sure he wasn't always facing to the left or right, I alternated.