I think the biggest thing is women need to be ok with the fact that their partner may or may not be a good support person, forcing anyone into an emotional role that doesn't suit them isn't ever going to end well. My DH was just as blind and confused as I was the first time around as neither of us knew what we were doing or had done any research... But the second time around he was a pro. The love and support I felt from him was amazing and there were so many times where even though my doula and a midwife was in the room it truly felt like it was just us, but that also says we had a great doula too.
I think you are right though, sometimes men just shouldn't be there but their partners need to be ok with that and find the right support person to suit them.

