I believe that if your meant to have a baby at a certain time it will come regardless of whether you're ready or not Maybe your baby isnt ready to come yet and thats why it feels like youre not quite ready?
Also, maybe try and forget that you planned to start ttc now because that can put pressure on you too. Remember nothing ever goes to plan unless its in 'the big plan!'
I really do believe that most big things are out of our hands so just focus on the here and now... it sounds like your right in the middle of some big changes that are going to definitely help you along in the future! You'll know when you feel the time is right to throw away those contraceptives!! Good luck! Exciting times ahead!!!