I'm not sure there is much you can do about genetics Children come in all shapes and sizes. My son is on the smaller side of things. I was slightly worried at his 5 month check, but as my husband said, "someone has to be in the 3rd percentile!"
I imagine it must be hard if people treat your children as older than they are and therefore expect certain behaviour/speech etc from them...it would certainly be a boring place if we were all the same size!
So I guess I don't have an answer to your question of how big is too big. If they are active, healthy children then hopefully there's not too much to worry about.
Bookmarks