Justine, I've read an article in Essence magazine about a 'condition' for want of a better word, the name of which I cannot for the life of me remember! But there was a personal story from a woman who said she felt utter despair as her milk let down, then she would gradually gain control of her emotions again and talk herself into realising that things were ok and she could cope. I can't imagine having to go through this every time I breastfed (particularly cause P seems like she's always attached to my boobs), so I really feel for you. You are certainly not alone - there are others out there who experience a similar thing.
Know that you're doing such a wonderful thing for your baby by sticking it out - I really admire you. Take care.