I think it would be different if it was an adult's party, or an evening affair. Then I would clarify that the kids were invited. FWIW I have been in a similar situation, where someone brought her fiance to a party that he was not named on the invitation but I had to face the fact that people so often come to parties as a family or couple group, and the only was around that is to stipulate "partners not invited" on the invitation. Which, for anything other than a girls night out, would result in my reconsidering the friendship.