Hmm I never had that problem, my son is determined to stand up! He just kinda 'hangs' it over the potty and off he goes
Such a man..LOL

Maybe teach him to hold it down!?