This is just a suggestion but if you want to be non confrontational you could take him when he's stressed and explain you want him to associate his Grandma with love and cuddles not stress and crankiness so you'll calm him down and give him back when he's happy. I use this with my Mum and Dad so they don't try and disipline my daughter.

Hope it helps