Caffeine is said to increase the risk of very early miscarriage as it is suggested that high levels of caffeine (ie 6 cups of coffee a day) can inhibit sucessful impalntation into the uterine wall.......but that is only in very early pregnancy.

So if you want to give up caffeine for the good of your health/baby, it would be best to give up/cut down whilst you are TTC.