It depends on the agency. Alot of them do a full-background check on thier sitters and nannies and will let you interview the sitter for placement. If she had passed a criminal records check, had a blue card and the boys seemed to like her I'd be ok with it (but I'd probably install a couple of nanny-cams and tell her about them to install a sense of paranoia in her rofl).
In a way your children are safer with a nanny from the agency because you can't run a background check on the neighbours (not legally anyway lol) and they might get offended if you ask to see thier blue card.
We're stuck in hard place here because I don't want the ILs to babysit until my children are old enough to rat them out if they do anything they shouldn't but if I get a sitter MIL will carry on like it's the world's greatest insult.