DS2 was like this from the start. I use to "dream feed" him. I would put him to sleep in our bed and after 3 hours I would put my boob in his face, he would smell th milk and start sucking. I believe in not waking a sleeping baby and he was alittle prem so.....
Also your baby is just sorting out how to feed, dream feeding might get him use to drinking when things slow down