We just had a weekend away in Ballarat. Stayed in a gorgeous B&B 2kms from the centre and it was so quiet!! There are lots of little towns nearby that you can drive to, but you still have all the major ammenities.
I'd also suggest Bendigo if you're looking to go a little further, or even Echuca/Moama and staying at Madison Spa. We'd have gone to Echuca/Moama again, but we wanted something different this time and hubs has never been to Ballarat.

Enjoy!