My bubs is only 5 months so I don't have much experience to draw on. Can you take a year's leave from work to give it a trial? My DH has cut his work back this year to see what it's like spending more time at home and having a bit less money. I believe you need to follow your heart and things will work out...it sounds like you know in your heart what you want to do.
xo