It's easy to do big handed stuff like ride a bike, drive a car, drink a coffee outdoors and so on. If you need to do something fiddly like do up a small button then they generally have to come off.