Ruby - I went the other way. After living in Townsville for very close to 20 years I moved to Cloncurry when my partner got a job out here.
As MissWingBat said - its more the streets in certain suburbs that you need to avoid. And certain parts of certain suburbs are a bit scarier than others! I do agree about Kelso and Rasmussen and some certain streets in Vincent but also some streets in Heatley are good to avoid. Some suburbs are much pricier than others and I really can't see justification in some of them. But - have fun and enjoy Townsville.
Bec