Heck FWIW neither of my kids were great eaters until they were nearly one.
My little guy is under the 10th percentile and no-one's ever said anything to me about him being outside the norm.
Don't mean to offend, but I think your doctors advise is way off base....like circa 30years ago. Please don't decrease his milk intake. Trust yourself.
he'll get it when he's ready.
Nate still only eats maybe 3 tablespoons of food at any sitting.