Yes I would, but we were only considering it because of remoteness. I was then really concerned with keeping her out of school activities up so that she had a wider circle, not just the kids at school.
When I was a kid a couple of my friends went to a local school with only 15 kids and both of them went through high school/uni successfully, so I guess it does depend on the child. That was a regular state school though.