im a kids swim teacher and tbh its a tough one!
do you just go to a pool and have a play? i highly recommend that you do if you dont
id be inclined to keep trying but using different tactics...getting him to just sit on the seat with you for one week, then move to sitting on the pool side etc..playing on the side or just joining in at the end for games.
that gives him some time to feel comfortable again.