We had the same problem with DD2. We tried the pram and the baby bjorn harness but it wasn't until I borrowed a pram from a friend that lets baby sit facing you, instead of facing away, that it really got any better.
The other day I managed to shop for around 3 hours while DD happily sat and watched me!