Yeah I've heard of this being related to exercise as well.
for the last 8 weeks I have been slowly losing my baby weight - sensible diet and about 30-40 minutes walk most days. I've really been taking it slowly because I was paranoid when I started that I'd bugger up my milk - so far no problems whatsoever, Toby hasn't noticed and my supply is fine.
Is it possible you're overdoing it a bit, or doing it just before a feed? Maybe slow down a bit and exercise right after a feed might help...