I kinda know how u feel. In my 2ng pregnancy I was the same. Hubby wasn't too bad, but I did feel rejected a fair bit. But I knew my body shape didn't really bother him either way. For him it was more work & the fact that when I started to show it hit him that there would be another baby & that scared him.
It's possible thats all it is. He was also spending alot more time with mates & infront of the tv - not in bed with me.
Ask if he's scared of being a dad. Or if he's scared of hurting the baby. It's more likely to be one of these two.