My first 3 pregnancies ended in m/c. I was also diagnosed with PCOS before we started trying so thought we were in for a long haul to even fall pregnant. I fell pregnant first try everytime, I just kept miscarrying and couldn't get any answers which is the hardest part.
I fell pregnant last year while being on the pill and I had a beautiful, healthy baby girl. I then fell pregnant again while on the pill 3 months after I had her and am currently 19 weeks pregnant with my 2nd baby who will be born a year after my daughter.
So please don't give up hope if you'd like another baby, PCOS is not a condition that will definetly make you infertile, it can just make it harder, or in my case I actually fall pregnant easier than most women obviously.
I blamed myself for everything I did or didn't do during my first 3 pregnancies, but the truth is I haven't done anything different for the two after that. I highly doubt you caused this in any way, especially with diet.