I've just done a search & haven't come up with anything to support that peas are bad for fertility. In fact I have found quite a few sites that say the opposite - any sites for nutrition & infertility I have found actually have peas listed under nearly every vitamin & mineral that you need.

ETA: Sorry don't mean to try & discredit the book, but having fertility issues I like to research things & want to make sure if there is some truth to it that I stop eating peas (or whatever it is in question ).