DS used to say 'for f*ckers sake' - in context too! (my bad). I just said to him "Mummy is wrong to talk like that, it doesnt sound very nice when we say those words". He stopped within a week.

I know the word shut up isnt a swear word, but I dont think it sounds very nice coming from a little kid. DS used to say it to the dog all time when he barked (again, my bad) but I taught him to say 'be quiet'. Now if I ever say shut up, I get told off by DS and told "Mummy, we dont say shut up. We say be quiet or knock it off".

We are not religious but I have taught my kids to say Oh my gosh, instead of OMG.

Just wanted to add - If you find it goes on, then maybe try talking about it in a non threatening way. There is no good yelling at a child for saying these words when all they are doing is copying what we are saying or hearing. Most of the time it is innocent.