It wouldn't hurt to look into it at the very least. It might even be something physical like his tongue that is easily fixed.

Being concerned is good - mothers the opposite of you drive me bonkers - there was a kid at my daughter's old school who was EIGHT and noone (ok, her big sisters could and they translated for her) could understand a word she said, the school had access to speechies, there were no excuses. And a distant cousin of mine was left until he was well past 4 without ever uttering a word and my grandmother kept trying to tell his mother something was wrong but she refused to listen.