I'm with helle, never minded people touching my kids - as long as they didn't stick their grotty fingers in their mouths... But that was usually my FIL, so I sorted that out pretty quickly! It's ok to simply not like it, but seriously, there is really nothing that your kids will catch from a simple touch of someone's hand to their arm or even cheek.

If you don't like it, I would simply say 'please don't touch him/her' and leave it at that.