My boobs just swelled from the minute I got pregnant. In fact I found it really hard to keep it a secret the first 12 weeks cos everyone guessed thanks to my lovely puppies. I started off as a 10C and got to a 12E (yep 3 sizes) by about 14 weeks when I went to get fitted for a maternity bra. Mind you I stayed at that size right through my pregnancy and probably until about 6 months after I had DS. They are starting to shrink back a bit now, I'm still breast feeding, but I am noticing that those E cups are a bit looser now.
This is exactly what I feel is happening to me. I've heard that people at work are talking amongst themselves wondering if I am UTD (and no, they didn't know I was trying!). The BB's is really my only symptom at the moment so it has to be that that has them guessing I think!