They really shouldn't have a problem with his nan taking him. You could always mention it when you RSVP and say that both of you are busy that day but if its ok if his nan takes him along. It would be a real shame for him to miss out. And it would be unfair to push it on your OH if he has anxiety issues.