My girls drain the dregs of my cups of tea (mostly herbal, sometimes I have an Earl Grey or other black tea), it doesn't worry me. I've actually considered making chamomile or something a regular part of their routine just to see if I can get them to bed any earlier than usual lol! I do rouse at them if I catch them sipping from leftover coffee cups (I drink a LOT of coffee and never seem to finish a whole one, it goes cold too quick and I end up making a fresh one!), because I make them really strong and they're bouncy enough already without getting a caffeine hit, but I don't mind so much if they drink a bit of tea. I try not to encourage it though, as I worry about them forgetting that tea is generally served boiling hot and scalding themselves with a fresh cup that I'm not keeping an eye on!
I don't think it's a big deal if it's an occasional thing - all things in moderation, imo