I think you should share whatever info your comfortable sharing

I also worried at DH work functions when people would ask if we wanted more "oh maybe" etc but now I'm to the point "yes, we've been trying for almost 4 years" it stops most of them in their tracks but hey I figure if they are asking they should be prepared for any answer.