when we lived on the farm, we had well over 15 in 2 months!!
Peppermint oil is supposed to keep them away.
If I were going to try to block the door I'd use steel wool as they're unlikley to want to chew through that, but that could be a pain depending how often you use that door.