Women with PCOS will often have no idea when they are ovulating, if at all, which makes conceiving hard as the whole system can be out of whack. Say for someone with your standard garden variety cycle, you might get end of period, then the gear up to ovulation with EWCM, libido increase, etc, then ovulation, then body stays in preparation for pregnnancy. With PCOS this whole process will kickstart, go in circles, fizzle out, limp along and just generally drive people crazy. It's really hard to track what's happening.
Pretty much sums it up. It just flippin' sucks!
Also for most women it takes a long time to actually get diagnosed and start treatment. So it takes longer and longer to conceive.