Oh Cherie, I agree. We weren't successful and I feel so much guilt for dragging DD around to endless appointments. It was less time I could spend with her doing fun stuff. But I always had things to entertain her and she was always really well behaved (but probably a bit older than your little one). Even now, when we have been finished for over a year, I mention a doctor and she says "mummy, is that the IVF doctor?"
It will all be worth it for you soon I hope.