We did it here too. For us, it worked out the most affordable way to get what we wanted. If we had bought established, it wouldn't have been in as nice an area, and it would have needed a lot more work. The only sacrifice is that it was a bit further out of the city, but closer to the beach so that was a win for us.
Not sure what state you are in, but here in SA, there is a government run agency, that gives loans to low income earners and families etc, and one of their loans is for building. The good thing with that is you don't have to start paying your mortgage until handover of the house, whereas other banks you start paying as soon as you settle on the land. For us, it meant that e could still pay the rent whilst the house was being built. So maybe worth a look into.

And as a few pp mentioned, we didn't build our forever home. It is our home for right now, and we know that within 10 years we will have outgrown it, but at least we got the starter home that suits us.