It's not appropriate and I would politely and non-confrontationally (if that's a word) say to him "I don't think that it's appropriate for you to comment on my appearance at work" and leave it at that. If he continues, take it to your manager and say that you have tried to address it yourself but it hasn't stopped and it makes you feel uncomfortable.