I don't know much about scabies in humans, but in animals it's called sarcoptic mange and I've seen it countless times when I was veterinary nurse. It's terribly contagious and can take months and months of treatment just to get under control. It can also take weeks to months after exposure for the symptoms to actually start to show. So if your bub got them from the great-grandparents you might worry for weeks/months to find out if he got them and needs to be treated.

I know it's a horrible situation, but I personally wouldn't take the risk I wouldn't even want him in the house either just because it sounds like it's a pretty serious infestation. Usually prolonged exposure increases the risk of getting it, but it sounds like the sheer number of creatures that might be in the environment would certainly increase the risk also. Perhaps if you go you could tell a little white lie that your bub has been unwell and it at a higher risk of catching infections at the moment and your doctor has suggested not letting others hold him? I know that seems horrible, but at least it wouldn't hurt their feelings and they would still get to see the him...