I say go for it, don't worry about anything negative anyone else has to say. There is always someone who will tell you that you are too young/too old, you have too many kids, your kids are too close together/too far apart. You just have to do what is right for you.

My old boss just had her 5th child at the age of 45, everything was fine.