Sounds the same as when I had mastitis Came on in the afternoon and at about 10pm I was shivering like crazy (this was still in the middle of summer!)
I would go to the Dr in the morning if you feel the same and temp still up. For me it started clearing up pretty quite once i started antibiotics. I always fed from the effected side first to make sure it was getting drained and just pumped the other side.

Hope your feeling better soon