IMO if we only had children when we thought we could financially afford to lots of us would never get there. I have one DD and am pg with no 2 I know that financially we would have been better off with just one and even better off without kids at all but we will manage i know we will.
If you want more and your DF is willing to have more than go for it. I am 1 of 5 kids in my family and i do not think i could have let my DD be an only child (not that there is anything wrong with only children etc) i am jsut so thankful that i have all my siblings as we are all the best off friends now that we are older. My DD has lots of aunts, uncles and cousins too.