I agree with Astrid that there's something else going on- seems like a small reason to stop talking to someone.
To me it sounds like it might be the people themselves who have the problem and not you - just because you have few friends, you don't have to take on whatever comes your way for friendship You ARE allowed to be selective about who you keep close to your chest and who is ok to remain an 'acquaintance'.
You have an 8 month old - thought about going along to your local ABA group? Lots of mummies and kids there from all sorts of backgrounds and you're bound to hit it off with at least ONE person! Also, soon it will be playgroup time and there is a few to choose from where you are.
The friends in my 'inner circle' nowadays are from my ABA group, one who I met through horseriding and fire brigade - there are so many people I know through these networks, yet only a few make it to be my friends that I place any trust in. Sure, I like heaps of the other people and we get along well, I just don't hinge on their approval of me
You ARE worthy, sometimes you just need to get the personality recipe right