I think considering how little contact you and your husband have with this cousin, it would be perfectly fine to use the name if you like it!! I'm having a bit of trouble with my mother atm because my husband and I have picked the name for DD2 (due in May) and it just so happens that the middle name we've chosen is the name of my mum's niece or cousin (I think it's her niece, my cousin, I'm confused haha), who she never had a good relationship with and hasn't spoken to in well over a decade. She's miffed because 'Oh that name will always remind me of my dead**** niece' but my husband and I like it and we're sticking with it, no matter what.

Maybe you could try it as a middle name instead, if that makes DH more comfortable with the idea?
I think at the end of the day, the name you choose will become your child's name and theirs alone, you and DH won't be reminded of relatives or people who have the same name because every time you hear the name, you will think of bub. I know this is true because my daughter shares a name with a girl I went to school with who was a real bully and made me miserable, but I never, ever think of her at all when I hear DD's name - it's all hers now.