I leave them unless I notice that they're rubbing but DH has a campaign against them and rips them out - it drives me nuts because he rips the seams as well grrrrr....
A few brands print them on the fabric instead of putting them on tags now...