You need to stop them getting under the house. I bet there is a nest under there.
I would treat them with front line ever 2 weeks & block under the house & get it sprayed to help under there.

Its probably only going to get worse with summer coming if they keep getting under the house.