Agree to a point with MD - plenty of times there are genuine reasons for needing to be early/late. But otherwise I think it depends on the situation and person.

Eg my brother and SIL regularly turn up early, with notice, to help. I don't care if they see me in my not yet ready hair, face and clothes. They help prepare then they watch DD while I get ready. Others will turn up and just watch while I'm trying to prepare stuff - like sit at the dining table and just watch and wait. That feels rude.

I start things on time, when I've said it'll start. If someone's late they can't get offended, unless they've given notice. If they just out of the blue turn up half an hour late and get annoyed that lunch has started because that's when cooking finished and I couldn't hold it, I think it's not really my fault - they knew when it started, I usually have a reason for when things start, and if they haven't even called/texted to say "sorry, x has happened and we're running late" I consider that their problem.