I have a sis that lives there - seriously I think the area is underrated. You are close to great beaches, there is a major shopping centre and good transport. Certainly there are less desirable areas too but I really think that is everywhere. Oliver's Hill and Frankston South have some pretty exxy areas with amazing houses. There are also other options close by as pp have mentioned - also Mornington, Mt Eliza etc.
FWIW too Margaret Lomas (a well known Property Investor) has spoken quite often that she feels Frankston is very cheap for the quality of the area, so anticipating some serious growth in prices. The council has also a massive development plan for the town centre and a new Marina, I believe - not sure how far down the track this is tho! If DH and I didn't have to go to the city regularly for work we'd definitely be moving.
Good luck on your house hunting - work out what works for you and your family and go for a look, maybe even consider renting for a while to suss it all out, if it is possible. I am lucky enough to live near the beach and would never go back. xx