This is something that really gets under my skin! I never just rock up on someone's doorstep, as I know I hate it myself.

Maybe you could do what some of the girls above said and not answer the door and if they say next time you see them that they popped over you could say "Oh, did you? Sorry but I was having a sleep. Just give me a quick buzz next time before you pop over to see if I'm up and about" or something along those lines.