Janet, I can't see anything wrong with the sort of tea you described, Stay away from mint teas and raspberry leaf teas, as they can bring on contractions, but most of the others are fine. I did find as my pregnancy progressed that my caffeine tolerance dropped rapidly, if I drink coffee my heart starts racing but I can still have a cup of tea every now and again.
Pay attention to how you feel for 30 minutes after drinking the tea and you should be able to figure out if you are having a negative reaction to it! Hubby is probably being overprotective.

Kerrie