Suse, my BIL (he's a non-Muslim Australian) refuses to wear shorts unless he's swimming on the grounds that the sun shining directly on his legs is hotter than if it shines on long pants instead so he'd definately agree with you.
Of course from skin cancer/sunburn perspective covering offers much more protection and it decreases wrinkles. If you ever meet a woman who has covered her face for a long time you'll freak at how few wrinkles she has.

ETA - yep I hang out at home in shorts and a t-shirt (or my pj's if I'm in a lazy mood lol). I usually keep an abaya (a long dress/robe)around to chuck over the top in case of unexpected visitors but sometimes I can't find it in the chaos and then I have to run around like a mad woman looking for it - it's a bit embarressing.