I was given some advise by my HCN the other day, who said to one at a time, cut a little of the tip off the top of the dummy. This will eventually annoy them more than satisfy them, and they will throw it away themselves.
I haven't tried it yet, as DS is due in July and I don't want her to be unsettled when I'm up all night with him, so I'm going to try when she is about 1&1/2. So I don't know if it works, but I guess you have nothing to loose, except a dummy!!!