In SA, (where I am) it depends on your council etc..

I had to get certified with the local council, (inspect kitchen and work practices) I used to work out of a commercial kitchen so I didnt have to worry about it.
They come out and give you a list of stuff that you need to be up to standard on, then you pay a fee, they come back and inspect and make sure you tick all the boxes. They can also come and randomly inspect at any time.
The council also provides free of charge a food safety handling course (basic food prep stuff)
I'm not allowed to sell products with fresh cream, raw eggs or confectionary (different rules/standards etc)

I have an ABN, but not registered for GST as I don't make enough to justify it.
I don't have anything on my FB page or website to qualify any of this, but I know I'm covered!

I don't do markets as I can't justify the cost of the insurance for a food stall (most markets wont cover food stalls with their "umbrella" insurance") I also dont have any idea of how much I would make/lose in a market situation, I do ok without doing them.