Thankfully, little kids don't understand when other kids are being mean so it doesn't bother them like it bothers us. I don't think you need to build a giant fence just yet. If the neighbours were going out of their way to bully your DD then I would be more concerned, but it sounds like it is only really when your DD goes to them. ATM your DD is seeking them out, but eventually she will work out for herself that they are nasty and stop wanting to play with them. My DS1 (nearly 4) has just started telling me which kids at daycare are nasty kids that he doesn't want to play with.
This won't be the only time in your life that your DD wants to be friends with someone you don't like. Sometimes you just need to be as supportive as you can and trust that she will choose the right friends in the end. Luckily at this age she is unlikely to be corrupted too badly.