My DS is the same, he will only drink milk from a bottle. He has one first thing in the morning and last thing at night (but not in bed)
He drinks water fine from any sort of sippy/straw/regular cup, but MILK MUST BE IN A BOTTLE lol.
I dont really know how to fix it, and TBH I am not too concerned, we did try all sorts of other cups and he wouldnt have a bar of it.
I guess when he is 3 or 4 I will deal with it, as its only for 5 mins max twice a day, at the moment, he is too young to understand that he cant have the bottle (his language skills are a little on the slow side) I am hoping that when he is older we can explain it to him.
We sacked the dummy at 5 months, I would imagine at 2.5, you may have a battle on your hands. Is he old enough to understand the dummy fairy? If not, I would wait until he is, I think that is prob the gentlest way.
GL