I had a fold up travel kiddy toilet seat that I carried around for DD (she was little and that age too). It came in it's own little drawstring bag and fit well into a nappy bag, or I would just put under the pram (I vaguely recall the brand being Tommy Tippee, but could be really wrong there!). Think I just bought from Big W or woolies.
I have one of these foldable seats for DS. I just really don't like the idea of him sitting on public toilet seats. I bought it BigW for $15. I just chuck it under the pram in a wetbag with a spare pair of pants.