We had a duckie-themed party for DD2 and I made some lovely duck biscuits iced in soft pastel colours. Every guest (young and old) took home a cellophane bag with 2 biscuits in it.

Other ideas:
Tiny box of sultanas
Bottle of bubbles (at this age they still need help but love the bubbles anyway)
Small bath squirter toy
Duplo minifig (can buy them seperately online)

BTW MrsB I *love* the sock idea!!