Maybe he also feels that he doesn't know what to do, even though you have your other DD's maybe he feels like it is all new and just lost the knack of 'babies'. When DD was born my DH had never had anything to do with babies before and everything he seemed to do with her was stiff and forced and he had no idea how to hold and comfort. It took him a while before he felt confident with her - I would say until she started to respond more and could start to support her head much better.

With the twins though he seemed to click faster as the age gap between DD and the twins is only 18mths and he still had those skills, plus he needed to be more involved as there were 2 to deal with.

Maybe he is just feeling a little lost as the nuturing instinct is more natural for mum's where dad's seemed to need to learn the skill and like most things, if you don't use those skills constantly you can lose a bit of confidence. I would try and get him to take on a specific role in her care ie: bathing as this then is one less thing you need to do, plus it gives him a chance to get back into the groove.