How do you know if it was something you ate that upset your baby?

Oliver has been pretty unsettled today, lots of crying (different kind of cry) and not settling. He also had 3 dirty nappies this morning and they've gone back to having those lumpy bits in it (he's been going only once maximum twice for the last two weeks and normally in the afternoon)

I had a bowl of ice cream last night, and another bowl of icecream about two nights ago... could it be that?