I think it is a lot to do with confidence too. If you are confident in your looks and personality then you will be more confident in a lot of aspects of your life. Confidence is reassuring and attractive. Whereas if you have self esteem issues then it would be easy to look at someone who you perceive to be better looking and attribute their success to their looks instead of to their general assertiveness and confidence. So people who suggest you got your job through your looks are likely to be just jealous and wish they had the skills and confidence to succeed in the same way you do.