DH and I felt similar to you and your DP. I felt I would regret not having more, he didn't really want more. But he told me he could not deny me another child if I might resent him because of it, and he didn't want me living in regret. So we had #3.
Now, I would like to have #4, but he is dead-set against it. But this time is different. I know his reasoning and agree with alot of it. I can see past not having another child. If we were to have another, so be it. It would definitely be an oopsie baby, but it would also be terribly loved, like our other children are, by both of us. But I am respecting DH's wishes this time and not going on about having another baby, because I know where he stands. Most of the time, this stuff is all about communication and compromise.