I was going to suggest oil of cloves too. You could suck on ice cubes too. If there is a bit of infection with it, try gargling warm salty water or mouthwash.