You are doing a really good job
Maybe hes getting enough form the feeds he is having.
If his nappies are wet and he is happy, alert etc (as much as a 4week old can be) then everythings probably fine.
If you are worried call your mchn or gp.
He may also just be having a fussy period. Its hard to tell.
Is he sleeping between the feeds?
My eldest child also started having longer gaps between feeds (5 hours at night from 1 week) which worried me. My younger child started just taking one side on a regular basis for abour 4 weeks which surprised/concerend me as well, but I just had a good supply and he was getting all he needed.