I use to think the same thing with the Baby Bjorn and my mum suggested a folded up muslin wrap, I ripped one in half so it wasn't as bulky and stuf that between his head and my chest.