Yeah, I would just attach him to the breast (in a sling or something if you can) and go with it. He will grow out of this in time (and you'll be on to the next thing, lol).
You're doing great, btw, sounds like you're an awesome team.