I'm with MR, Maya etc. Pop over with some morning tea, have a chat and get to know her. Don't be fussed if she looks frazzled, it's likely she's embarrassed that someone's busted her in her pjs/bub screaming etc. And yeah, if she's looking stressed, offer to mind the kids while she has a shower, or goes shopping (or offer to do her shopping for her). But TBH, I personally would need to have met someone more than once to be able to leave my DS with them, even just to have a quick shower. But as she gets to know you, she may well feel comfortable with that, and really appreciate it!

Good on you for wanting to help It's nice to have neighbours with kids too, I hope you get along with her.