Like Catherine I was more concerned about me than DS losing his day sleep, but he dropped his just after 2. Since DD had hers until 3.5 I was feeling a bit ripped off lol.

Seriously though, babies don't read those books, and none of them were written specifically about your child.
My DS is very much like your little guy. Even half an hour in the day pushes his sleep back hours. Sometimes they really do need it, or it's unavoidable (like if you're in the car) and those days I just go with it. My DS is known to fall asleep in random places when he's tired - sometimes at 4.45pm. Lucky for me I have learned that means he's good until the next morning, but trust me I understand the frustration.
One of the best pieces of advice I can offer you is stop watching the clock. He'll sleep as much as he specifically needs.
I found that when I followed DS his patterns settled much more. He now goes to bed at 6pm. Unheard of for a kid who didn't go to sleep before 11pm until after he was 1.