Personally, I'd not tell...
You don't owe them anything & having worked with Agencies over a 9 yr period (as it was way better money) I used to leave & travel @ the drop of a hat & then come back & call them again, they expect it & don't think you need to tell them anything unless it was work that could cause problems...