At four weeks my DD was only up for 40 minutes at a time. Now at 6 months she is only up for 1.5 to 2 hours.
I would think that 2 hours at 4 weeks is too long and he is getting overtired. It is so hard to know but you will learn his tired signs and what he needs and wants over the coming weeks.
Good luck.