Thanks so much for your comments.
I do want to move away, and if I did that due to the distance (about an hour & a half) he would only have them every 2nd weekend. However as a compromise I could move closer to where I want to live and they could still go to their current school (by private bus) and potentially could still see their dad regularly either 50/50 or some other arrangement.

It's interesting people haven't recommended 50/50. We have a good relationship and at the moment the kids stay with him every second weekend and also 1 or 2 nights a week. The kids are being shuffled about a fair bit which I worry about, I thought one week on, one week off actually might offer them more stability/routine than our current arrangement.

I wish I knew what to do.