I don't know specifically about baker's sperm but our gyno did tell us that men who work in hot environments (he used examples of taxi drivers cause they sit down all day) often have reduced sperm quality. My husband is a plant mechanic (works on mining equipment etc) and works in very hot conditions so it kinda applied to us too. I don't think it should be a huge deal though, doesn't appear to be for us but if you are concerned enough and can convince your DH, you could always have a sperm analysis done for peace of mind. You can organise this thru your gp.

Best of luck!