My DD2 is nearly 4 months and won't be wrapped either - she needs her hands free to suck and won't sleep if she can't access them so not only are they cold, they are also wet! DD2 sleeps in our room and the temp is always above 17 degrees overnight.

I read somewhere with DD1 that babies hands are always cold and it doesn't bother them and it doesn't seem to wake her or anything so I have just left them as is. Mittens aren't recommended so there isn't really much else you can do.