You'll be right - he'll go back because that's where his milk comes from
You're right, 'bonding with Dad' isn't actually necessary right now. Dad's time will come, trust all of us here! Right now, biologically, evolutionarily and psychologically, you are IT. It's ok to be IT, it's ok for Dad to take a back seat while you exist for your boy. YOU are the one keeping him alive and that's all he cares about. Look at it from your DS's point of view and it might seem clearer, and you can all let yourselves off the hook