I ALWAYS leave it! Once I was at a baby shower where everyone bought a plate, and this woman left early so she went to the buffet and started packing up the leftovers of what she had brought to share, while people were standing there still picking!