I agree, it's totally up to the child and the family.
I dropped swimming in the last year (they still do other stuff though), and I didn't realise how tired and over structured they (and I) were. I love not having to be somewhere every afternoon after school, and they have benefitted from extra down time. So, each to their own I say.