Well yes, I do think it's rude to wear sunglasses when you are talking to someone, especially when you are inside.

If you aren't talking to anyone, just wandering around the shops or whatever, it's not rude, but I do think it looks stupid.

I find it very offputting not being able to see someone's eyes when I am talking to them. Even if it is bright outside I will automatically take off my sunglasses when I am talking to someone so they can see my eyes.

If it's someone I don't know well and they are wearing sunglasses and I am not, I don't feel the conversation is on an equal level - I will put my sunglasses back on because I feel like they aren't putting as much effort into communicating with me, like a subconscious self-defence mechanism.

I don't believe the view that "if you don't care what people think you can do what you like" approach is always appropriate. Sometimes it is good to be independent of thought and action, other times it's selfish and just plain bad manners.

ETA: Before I had my eyes lasered, I used to have prescription sunglasses and prescription glasses and I never found it a big deal to switch glasses over.