Sounds like a low grade UTI...urinary tract infection.
Cranberry juice has been clinically proven to help.
It may be best to go for the sugar free cos the stuff is high in sugar or you can get cranberry tablets which work just as well. I personally perfer to use juice than tablets.
I keep a bottle of juice in the pantry for such emergencies.
Avoid antibiotics if you can as they will knock off all the good bugs in your system as well as any causing a problem.