:
Infectious diseases, such as chickenpox, can affect male fertility in a couple of ways.
First, a high fever associated with the illness can temporarily decrease sperm production. Typically, sperm production returns to normal within 90 days after the fever ends.
Second, infectious diseases may cause inflammation of the testicle (orchitis). This may result in testicular shrinkage (atrophy) and infertility. However, orchitis is most often associated with mumps or infection of the drainage tube of the testicle (epididymitis).
The second is more likely to be from mumps yes, but there still is that chance it can happen from chicken pox. And to me that minimal chance is big enough for me.