Question for all of you wonderful ladies in the know!

Since yesterday my supply has increased even more than it was. I think it was due to being in hospital and not being able to express for 5-6 hours, because since my boobs got so engorged then, they keep getting SO full and hard and sore. Feeding currently only just takes the edge of, but they'll still be hard and sore. When I was pumping I didn't get enforcement, maybe I'd be a bit uncomfy if left for ages, but now it's all the time! So my question is (may seem silly) is it okay to just let DD feed from my boobs and not pump the rest out? I don't want my supply to keep increasing, so I'd rather not pump the rest of my milk, but is it safe to just leave them to settle themselves and their supply, and just let Isla drink what she needs? I had ante-natal mastitis and it was not nice, so I don't want that happening! But I assume people who don't bf just let their boobs do their own thing and their milk will just go away without any problems? So do I need to be pumping after I feed or is it safe to just let my boobs sort themselves out? (And will they?)