I had one of these too. I can't remember what I did with it, but eventually it went away. And they hurt like snot.
I'd call your GP and make sure you haven't got an infection.