I have pcos (poly cystic ovarian syndrome) which means I have heaps of cysts on my ovaries......I realise that ovaries are different to a uterus, but if the doctor wasn't worried about it then trust him/her. If there was reason for concern, I'm sure your dr. would be onto it.......
I know it can be hard not to worry...but try!