Until about 6 months DD2 would just sit in the pram poolside. The edge of the pool goes up (so not a chance of the pram just rolling in) and when I took DS on Tuesday mornings there was rarely another sole even there.
Obviously it's not the same everywhere, because I still have to get in with DS at 4, and yes he can swim perfectly fine on his own. We have to until they're 5.
Now a GF watches DD2 for me while I take DS. She loves going there, and I get some time with DS.

Some mums in other classes have their littler babies in a float thingo while they are in with their toddler lessons, only I'm not a fan of that as it's too hard to give attention to both.