I was gonna say! What is a 5yo doing just popping over to neighbour's houses without his folks knowing? Wow!

Definitely speak to the parents. How awkward for you!
What if something happens? Betcha they'll blame you for it.... always the way!