No I don't think you should be unduly worried. I can still go a long time in between feeling movements, doesn't mean they aren't happening though.

Much later I would still worry when I hadn't felt many movements but that would be because there wasn't much room left in there. I used a kick chart in my first pregnancy towards the end and was surprised by how many movements actually happened, that I wold normally have missed.