Awwww hun I know just what you mean, Shel got really upset when I'd put her hand on my belly but she couldn't feel Jazz kicking, and then she'd get angry at me for telling her when Jazz was kicking. I think they just get a bit felt out when everything is happening to us, but don't worry too much, because even though he can't feel it now he's going to be so excited when he does actually feel it, and it'll be a great moment for you both.

ETA:
And then you can spoon him at night and let bubs kick the cr@p outta him
PMSL I did this to Shel too, in the last few weeks the kicks would wake her up!!!!

Oh, and I'd say he'll be able to feel your bub in the next week or so, from what I remember it was about 22 weeks that Shel was able to feel some pretty strong kicks.