I wouldn't be ok with my child being that far away, so you are not alone.

my 3.5 year old has only stayed at my mum's by herself once, and once at another friend's just ten minutes away (and I picked ds up at midnight, still knew she was asleep and ok).

when preparing for the sleepover at mum's, dd and I stayed there one night to see how she would go.

my mil has also left many hints, but I'm not comfortable with the idea yet, so just keep deflecting the suggestion.