I don't wear shoes inside, either. Except for my thongs when I'm cleaning and don't want my feet getting all slippery on the wet floors.
And yes, it bugs me that nobody else takes their shoes off when they come inside, even DH tromps around in his Chuck Taylors or *eeek!* work boots all day and it's like, 'Umm, the baby is crawling through that big pile of dried mud you just tracked indoors... thanks!'.