I wanted to know the gender of all my kids and DH didn't. That's how we rolled for all our babies. I am very good at keeping a secret. So if you DH can keep it a secret, no reason he can't know and you not
I am not a fan of gender specific clothes or toys either.
After two boys, people assumed I would be pink mad. Quite the opposite, I love bright and vibrant colours on all my kids and not just DD, she plays with dinosaurs and cars and I have not purchased a single doll other than Jemima who was DS1's
Sometimes you wouldn't even know I have a girl in the house as we don't really have any gender specific toys, a few that people insisted on buying her for her first birthday just.
I am with feeb, so many great clothes on the market that non gender specific, I love it!