Is he healthy, developing and reaching milestones?
Those charts are for mostly ff babies, and it is different, especially as they get older. You may find he's more like 30th percentile (total rough guess!) on the WHO charts. I know that was the case for my DS at that age.

If he's actually losing weight - and not counting if he's unwell, because that can happen then - and lacking energy, not developing/reaching milestones etc, etc, then there may be cause for concern. If you're unsure of what your doctor is saying, you could always find another to get a second opinion, too. Perhaps talk to someone with more breastfeeding knowledge?