Not really for me. I've done the preg/BF/wean thing twice now and I was pregnant when DS weaned but the only time I've gone down was when I lost weight, but that had more to do with my back measurement than my boobs IYKWIM?
However, I've not really gone up a lot of sizes either.