At his age i would tend to more ignore it. If he doesn't hear it again too much, he'll forget about it. I too let my kids know when they were a little older that it wasn't nice, and they'd let me know if I slipped up!!

Going on about it, or punishment, will give the swearing more meaning and oomph. He'll love it even more, and say it just to p1ss you off!!!