thats ridiculous, 4.1 is a normal sized baby.
Babies are born puffy cos they are born with extra fluids to compensate for the first few days before the milk comes in, thats why they all lose weight in the first few days/week and the puffiness goes down.
I think you can safely refuse the Gtt, maybe compromise if they are really getting silly by saying you'll have a GCT.
good luck!