I don't think your being silly at all. We are the same here. And ours don't have to go on a plane to go! I think you should just not give excuses simply say.....We (you and DH) are not ready for her to go away over night and think she is to young to be so far away.
Maybe ask your DH to tell her if you cant? Somehow I would also put in there that it makes you feel uncomfortable they keep asking and that you will let her know when your ready if and when that time comes.
Other option is send it to her in an email. I think it may come down to wording. Point out you think its great they love her so much and want to spend time with her, but as her parents you simply feel she is too young and is not ready for such a stay. Also our dd is 3 in Jan and she has never really been without us either and my IL's only live literally down the round from us lol