If it doesn't seem to affect you then I wouldn't be overly concernced - if I followed ALL the "do's" and "don'ts" of pregnancy then I'd be a decaffienated mess I do love my cup of tea in the afternoon.
If you've been taking it til this stage without noticing any ill effects then I'd hazard a guess you'll continue to be fine with it, just watch the quantities.

On the other hand, if you run overtime for a week, then you might be very glad of a few strong cups of peppermint tea!!

I know it's not the same, but chamomile tea is sovereign for everything - really boosts your auto-immune system, apart from the other soothing benefits.

Kerrie