Definitely don't panic. DD3 definitely slowed down/got less violent with kicks and gave me quite a few anxious moments. I went to get monitoring after a paticularly quiet period. She was fine, but the midwife sent me home after monitoring telling me to come back if I had any more concerns at all. You won't be wasting their time and they would much rather you be safe than sorry too, so go and get checked so you can relax.