I try for 3-4 weeks before the party if I know that there are other things on locally that may clash. And I also think that other parents might be like me and hate getting an invite a week or a week days before the party - trust me it happens! DD1 got a party invite once only 4 days before the party! And of course you can't say no they can't go because they are so excited to get an invite. My rule of thumb when they are little like that is only their classmates. Unless you are particularly involved with this other child, then I wouldn't worry about inviting them too much if you didn't have to.
Bookmarks