My DS's belly button did the same thing. The midwife said it could continue for up to 3 weeks after the stump falls off.

As long as it doesn't smell offensive, isn't red or raised or shiny then nothing to worry about.

Spring