Hi,

You need to remove some of that milk - and you are right - the simplest approach would be just to feed your baby. I doubt that he would be too confused at this age (has he been a reluctant weaner?) Otherwise, you might try to express some milk in the shower, by hand.
Use ice packs on your breasts to reduce the inflammation.