We too were looking at Xavier if we had a boy, pronounced with the Z sound, but a cousin used it about 3months ago for their baby. They spelt it Zavier as they thought it was pronounced X-avier
I don't think you will have too many problems with people spelling it, if anything it might be the pronunciation