It is likely to be very different if you're not around, yes. You'll find they do go through periods of greater need for attachment, but he'll be able to transfer that to his daddy (or others) if you're not around.