I don't sort colours. And I have never had an issue. I wash new things alone once, or twice if they need it, first.

Clothes, towels, sheets, nappies. There is only me and the 2 girls though. Nonetheless, I think you should sort less and never wash less than a full load.