I wouldn't, but at a certain age (about 7m or so) I find the koala hold in the HAB works wonders. Its also similar to a hold you would use in a ring sling where they have one arm out, and one arm in, and they are kinda on your hip holding on a little more.

Have a play wiht your HAB and see how you go.