From what our GP explained it as, its a part of the umbilical cord/navel that doesnt heal properly. She used the term proud flesh.

They usually give you a cream to put on it, and it will decrease dramatically fairly fast, worst case scenarios are silver nitrate (I think, its been a while) to make it go down.