I think its totally normally to feel how you do especially when it took longer then normal to get there in the first place. Soon you should feel movements of bub and this will be very reassuring as then you wont need an ultrasound to tell he/she is ok. Can you go to an AN appointment and have a doppler? Just for piece of mind. At 16weeks your health provider can almost always find it and hear a nice strong heart beat.
Unfortunately the worry will probably be constant until you have he/she is your arms (and you still worry after then) try and enjoy been pregnant as much as you can as if you don't you will look back on your pregnancy and realise how you didn't enjoy it as much as you could have because of the constant worry that something could happen. Its easioer said then done.