Well I don't know where you work, like if it's a church or something then it probably is a bit daring, but if anyone said that to me I'd laugh because it was a joke and jokes are funny.

Their responses seem a bit drab.

Don't worry. I thought it was funny.