I kind of had the same dilemma.
Work mates, friends and family and we decided after encouragement from some lovely bellybelly ladies to go ahead and have a bbq at a local park and just invite them all. I was worried about inviting too many people but apparently more and more people are doing something different these days rather than the traditional sit around and play games party.
We are having about 50 - 60 of our closest friends and rellies, guys and girls for a bbq at a park for a few hrs of chilled out fun. The guys will bring a footy, ppl will mingle in their comfortable groups etc.
You could do something like this if you weren't concerned about numbers.