I would risk the colds (you'll find he won't get any more than if he doesn't go swimming) - swimming is super important now to learn if you're in any way going to be near water during his childhood and teen years. The sooner the better. After about 6 months they start to lose the dive reflex - it is infinitely harder to get a child to put their head under water and feel nice about it if you don't get him/her doing it by 12 months (I'm being really generous with time frames here, because often parents don't like to hear that they've left it too late). It's primarily about safety and confidence in water. By 18 months, my kids have known how to turn around and swim back to an edge in a pool. It makes swim time fun, and also gives me a few moments more peace of mind.
We didn't start swimming lessons for DD until 13-14 months (for various reasons). She went under during her second lesson and has no issues with it. She looked shocked the first time, but didn't cry or gulp lots of water either. Do you have the option to pay as you go? At least to start with until you know how it's going to go.