I usually say "that's a shame, I think I'd make a good friend" and walk away. I don't usually give that much attention