I can't offer any advice other than to say go with what you think is best for you DH & your family!
Hope all goes well with TTC for you