I have done it, and we went on to keep our other property as a rental, but have almost officially sold it now (waiting for the money to hit my account in 2 weeks time )

The timing sucked construction started a LOT later then it was supposed to and they poured the footings when I was still in hospital with DD after she arrived. So it put a huge amount of stress as I wasn't working for as long as we had planned and it really threw a spanner in the works. But we got there in the end.

I would do what you can to reduce as much of your outgoings as possible, you should be able to pay interest only on the construction loan while building, and see if there are any changes you can make to your current mortgage to make it easier. Best to go in and have a chat to your home loan manager or bank manager, they can really help out, and best to do it upfront rather then when things get tight or tricky.