A friend of mine went through this last year. They already had 2 girls, and he desperately wanted a boy, and she didn't want any more children. After a few months they decided to have another baby. It turned out to be another girl, but they are over the moon, and wouldn't change anything now. They too have a 3 brm house, and just make do with what they've got.
I think if you're unsure, then take some more time to decide if this is what you both really want.