Yep, me personally I would look into ways to help DH's sperm quality.
There are supplements, as mentioned, that can help.
Also lifestyle changes - ie, diet, exercise, reducing alcohol, stop smoking (if he does), etc
Work out a plan, give it a few months to see if there's any effect, then look into IVF/ICSI.

Good luck!