I dont know what to do though... should I trust my instinct that they'll be crap and not go back at all? Or should I just go back, work my butt off, see how it goes, and if theyre being crap then resign?
Tough call but if it was me, I would go back and see how things go. If it really is that bad you can always resign but at least you can say you tried.

Good luck with your decision.