I go to storytime at the Library, playtime at the play cafe & often to the the pool each week. On these days he just sleeps at a different time. They are adaptable & when tired will sleep. Usually DS always misses his morning sleep, but I make sure he's well fed before leaving where we have been as he always falls alseep on the walk home then is into bed. Just usually means he doesn't have a big arvo nap, just a small cat nap later in the arvo......you do need to get out of the house at some point.
I try & do most things in the morn & be home in the arvo as that's the big sleep.