Ah sucks doesn't it? Sometimes I'm sure DH does it badly so it seems like I'm the only one who can do it.
Though it has worked out well for us as at least I don't have them screaming for daddy when he works 300km away.