Belfie - there are a few thoughts here...
I only ate back the calories if I felt hungry, which I could probably count one the amount of times on one hand. I can't justify eating if I'm not hungry .
Some people who choose not to eat back those calories burnt during exercise and who feel hungry later on, yet decide not to eat with the thought they will lose more weight, run big risk of entering starvation mode = body holds more in response.

Have I kinda answered your question?