AJ, I am in a similar situation at the moment also. I am opting for less stress and job satisfaction. I have applied for another job with better hours, closer to home etc hoping that it will help us along with TTC as well.
I would think that high stress levels, BP etc would have some amount of negative impact on conceiving.
For me, getting out of the stressful environment I think is going to help a lot. Everyone is different and I guess deals with stress differently so I guess all I can say is that you need to put yourself first and do what is best for you and DH.