A cat or a snake. Mice hate both of them, something to do with the smell. I think there is something available that mimics the smell of cat pee that can be used in areas that mice frequent that deters them even if you haven't got a cat there. Without it actually stinking like cat wee all through the house.

I'd also set traps regularly especially in known hideyholes.