I would go with what Olive said and then some. I'd involve the whole family in a separate swim session at a pool with a hydro pool (if they don't have dumb restrictions on age). That way, the water temp is not an excuse to not get in and it's pleasurable again.
DD hasn't taken to the pool the way DS did, but that's because we interrupted her swim lessons when we moved away from the only school I loved for baby swimming. I have since booked a private swim session at our local indoor (it's a 15m pool that's run by a committee in a country town) one night a week. DD is getting her confidence back (esp the last few times) since swimming with all of us in the pool, and DS gets an extra sesh in the pool on top of his lessons that no longer involve me (he's 5). DS is a gun in the water and going to the pool with him was always a fun prospect. By the time DD was born he was able to play near me and I could concentrate on practising what we'd done in lessons together. That's a combination of age gap and the swim school (not your typical YMCA program), and it ties in with developmental stages.
We just haven't spent that family time with DD since January, so it's taking longer to get her up to speed. The more she spends time with DP in the water the better she is, too, or she ends up clinging to me too much (I don't carry her, she has to hold on to my shoulders by herself). She improves when I get a chance to swim off with DS and do fun stuff. It helps so much to model having safe fun, showing them that it's enjoyable and we're not afraid of getting in. Not saying you're afraid, but modelling water play is really important, whether you have a pool or not.
I wouldn't force the issue of getting in the pool without you, that sounds like a disastrous plan. I'd just go back to getting in the pool with DS and putting the pleasure back into it, but I would also suggest that this be regular and frequent - hard to do when it's not scheduled in and paid for, like term fees!
DP wasn't initially keen on our swim booking, but I talked him round, explaining that by making it a permanent booking it was OUR night to do a family thing together, because everything else is pretty fluid in our family, and there is a place for some regularity.
A family play swim could take the mission out of swimming with the kids