Back to the initial question - yep, it does hurt for a bit and then it goes away but then it comes back when they start to teeth. I read somewhere it's something to do with the saliva and that if you rinse your nipples after a feed it is less irritating for them... haven't done this yet so I can't comment.
I never had a problem when DS was teething. Only thing that got me was the occasional bite, so don't have it in there after they finish feeding, OR when they are in the middle of a tantrum!!